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CHAPTER I 
INTRODUCTION 

The Probleia 

**It is becoming increasingly obvious that 
the full potential of information processing 
will not and cannot be realized until computers 
and their applications can be managed reliably 
in an economic sense (Weinwurm, 1968, p« 329).** 

Weinwurm’s above comment at the 1968 National Conference 

of the Association for Computing Machinery could well be taken 

as the underlying imperative for this research. Computers have 

had great impact on our economy, on organizations, and on 

individuals in the past fifteen years. They have been cursed 

and praised. The literature of virtually every discipline 

contains increasing references to computers. 

However, one recurrent theme that keeps appearing over 

and over again when computerized information systems are under 

discussion is the failure of organizational managements to 

capitalize on the potential offered by computer technology. 

Many accounts of such management shortcomings could be cited, 

and some will be cited further on, but Reynolds (1968) summed 

up the essence of all of them quite succinctly when he said: 

**The common complaint among people who must 
plan for and manage the development of computer 
program systems is that the products are almost 
always finished over budget and late, and they 
hardly ever do what they were Intended to do (p. 334).’* 
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John Diebold (1969), a pioneer in the information 
systems field, has predicted that **by 1985, the computer will 
have become central to the nervous system of the corporation* ** 
Diebold (1969) has also pointed out that the **share of personnel 
and software cost in the total ADP mix" has substantially 
increased in the past five years to the point where these costs 
often amount to twice the annual hardware cost* Both of these 
factors lead one to the conclusion that the management of 
management information systems (MIS) should be the focus of 
much attention and research. 



The Need 



Before proceeding to a statement of the purpose for 

the present research, the following rather extensive quotation 

from Weinwurm (1968) emphasizes the need for the type of research 

conducted by the author and reported in this thesis: 

"Of the explosive and unremitting change that is 
characteristic of information processing there 
can be no doubt. Nonetheless, there is a great 
difference between noting the pace and dynamism 
of the field and asserting that, as a result, 
management research is bound to be wasteful-*- 
no matter what the investment in it, regardless 
of the probability of success, and whatever the 
savings that may result. The empirical evidence 
that can be marshalled in support of such a 
compound assertion is rather meagre. Indeed, it 
is doubtful whether a deliberate policy of 
investing in management research on a scale that 
is in some substantive sense proportional to the 
magnitude of technical development has ever been 
seriously attempted. 
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’’Similarly since all possible innovations and 
contingencies obviously cannot be anticipated, 
learning by experience is a necessary concomi- 
tant to any technological advance. Adopting 
’trial and error’ as a governing policy is 
another matter altogether. There is consider- 
able reason to believe that the real costs of 
information processing ’trials’ are far, far 
higher than is generally recognized. In fact, 
due to the ingenious ways by means of which we 
have learned to obfuscate learning inefficiencies, 
the real costs are very likely indeterminate. 

In any case, no serious attempt has ever been 
made to demonstrate that ’trial and error' is 
the most economically effective— or the most 
expeditious— way of advancing the state of the 
management art; perhaps because such a demon- 
stration may well involve contradictions of a 
rather fundamental nature. 

’’Finally, there are undoubtedly rather basic 
similarities between information processing and 
any other process that involves men and machines. 

At the same time, these resemblances can be 
carried beyond the point of supportability . 

While the general validity of certain principles 
of good and effective management can be assumed, 
information processing in many respects differs 
from the manufacturing processes upon which much 
of our management knowledge is based. To the 
extent of these differences --which are expecially 
evident, for instance, in computer progranming-- 
literal and indiscriminate application of tradi- 
tional management principles can on occasion evoke 
rather preposterous management policies. Hence, 
the critical issue is not whether well-established 
management methods are entirely relevant or 
whether completely new variations must be developed, 
but which combination of the old and the new is 
most appropriate to and effective under each 
circums tance. 

”ln any case, whever the ’truth* , collectively, 
lies, one would expect these differences of 
opinion to be discussed by the leading experts, 
with the support of a reasonable assortment of 
facts instead of assertions, in and through the 
formal meetings and journals that the relevant 
societies provide for such purposes. That, 
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after all. Is Che professional and scientific 
way such issues are resolved. Regrettably, 
very little if anything of this kind has 
happened. It is true chat there have been 
hundreds, and probably thousands, of discussions, 
symposia, articles, tutorials and Che like, 
during the past 10 years, but nearly all have 
been on a transparently superficial, often 
amateurish and promotional level. And one 
finds it difficult to hear of comprehensive, 
comparable, or reliablyanalyzed experience- 
data«»let alone Che management measures and 
standards that derive from such data— anywhere. 

"I believe Chat we cannot abide this situation 
any longer. It is high time that Che management 
of information processing was brought into the 
mainstream of professional discussion; i.e., 
that theses be formally advanced and supported 
by facts in a respectable manner; and that 
this be done 'on the record' so that others 
will be stimulated to counter in Che same 
fashion. " 



"Vfhile the lack of a generally available, 
accepted, or applicable management methodology 
is a serious problem in nearly every area of 
information processing, I believe most 
experts would agree that the greatest con- 
fusion and the most critical need is in the 
management of computer programming— in the 
broad sense of the term. Moreover, software- 
related problems— by every present indication- 
will increase far more rapidly in their scope, 
con^lexiCy and economic impact than will their 
hardware counterparts during the forseeable 
future (pp. 330-331)." 



Purpose 

The purpose of the research detailed in this thesis 

can perhaps best be summarized by the question: 

What organizational and procedural factors are 
correlates of success with MIS projects? 
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Ralslng such a question and pursuing its answers are predicated 
on the hypothesis that there are certain organizational and 
procedural variables which have a meaningful influence on the 
success or failure of a given MIS project, and that these 
variables can be identified. Further, upon identification, 
such variables can be manipulated in certain ways under certain 
conditions to enhance the probability of success with a given 
MIS project. 



Definition of MIS 

Before going further, it is necessary to define just 
what is meant by, a "management Information system" since there 
does not appear to be much consensus on a definition among 
either authors or practitioners in the field (Dickson, 1970). 

One can arbitrarily designate any system or method that provides 
information for the decisionmaking functions of management as 
a MIS. A great many sources of management information beyond 
the scope of the usage in this thesis would thus qualify as a 
MIS. (For example, the Wall Street Journal .) However, the 
term MIS has only come to be used extensively since the advent 
of electronic computers for processing information to aid 
managerial decisionmaking. In fact, it would appear that a 
great many people equate MIS and the computer. 

It is this latter equation that the author considers 
to be incorrect. There have been, and are today, many kinds of 
computer applications which should be classified as data processing 
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applicatlons as opposed to MIS applications. The perennial 
favorite, when such a distinction is raised, is a payroll 
application which is merely converted from one form of process- 
ing (manual or punched card) to some other form (computer). 
Another example of a non-MIS computer application would be an 
inventory-control system which merely does on the computer what 
stock clerks were doing with card files, and nothing more. 
However, the inventory-control application could become a MIS 
if upgraded to the point that summarized or analyzed data were 
made available to management for the express purpose of making 
decisions on such questions as balancing workloads, make or 
buy, and optimal forms of distribution to minimize cost and 
maximize service. The last example might be viewed as the 
threshold for qualifying as a MIS application, since some of 
the kinds of decisions cited could be programmed and handled 
by the computer under certain circumstances. The important 
distinction, then, is that for a computer application to be 
considered a MIS application, a manager at some level (not just 
the corporate executives) must be receiving information from 
the computer which aids him in making decisions within his 
domain of responsibility. Such decisions could be of either 
a planning nature or a control nature, or both. 

The Focus 

Three additional points require elaboration with 
respect to the stated purpose of this research. First, the 
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focus of the research reported here, or ‘‘element**, using the 
terminology of Lazarsfeld and Menzel (1961), is the MIS project. 
This is a distinct departure from previous empirical studies 
of various aspects of computer usage in organizations. Generally, 
the element studied has been the overall organization or some 
segment of the organization. For example, there has been 
considerable attention devoted to the impact of computers on 
the organization (Shultz and Whisler, 1960; Myers, 1967; Reif, 
1968). Another approach used in studying the organizational 
Implications of computers has been to use the information systems 
function, or “computer complex**, as the element (Reichenbach & 
Tasso, 1968). This latter method involves trying to evaluate 
the impact of various organizational variables on the effective- 
ness of the overall computer effort. 

The assumption here is that if methods can be found 
to increase the probability of success with specific MIS projects, 
the overall success of the total MIS effort will logically follow. 
While this is a strong assumption, it is further assumed that 
an organization doing a good job on a project by project basis 
will not completely neglect long-range MIS planning and integra- 
tion. At any rate, it is believed that more meaningful hypotheses 
can be posed and tested for the individual MIS project than for 
the entire MIS effort of an organization. Further, there is 
need to evaluate the effectiveness of given practices in more 
discrete situational contexts than the total organization. 
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since project characteristics may possibly vary in such a way 
as to call for differential treatments. 

The second point requiring elaboration follows 
directly from the definition given of the MIS project, and is 
related to the point about project orientation. As noted, most 
previous studies have dealt with the total organization in 
looking at MIS success, and in doing so have encompassed an 
indeterminable amount of information system development 
activity that the author would not consider to be MIS. This 
has had a confounding effect because the requirements for 
effective management of MIS activities appear to differ in 
some respects from the requirements for success with non-MIS 
activities. Specifically, it is the author's contention, 
supported by the data collected and analyzed in this thesis, 
that development of information systems for decision-making in 
the various functional and executive areas places a greater 
premium on involvement and integration of functional managers 
and executives than does development of non-MIS computer 
applications, which may have evolved fairly directly from 
existing procedures. This view is supported by several authors 
in the field (Canning 6t Sisson, 1967; Ditri & Wood, 1969; 
Lipperman, 1968; Reichenbach & Tasso, 1968). 

The last point above has particular relevance when 
viewed in the context of the future prospects for computer usage 
in organizations. Many organizations have about come to the 
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end of the road on non-MIS computer applications, and are, 
therefore, looking to the opportunities offered by the laore 
sophisticated and complex MIS applications. A report by 
Scientific American , entitled **The Computer Market" (1968), 
showed that only 39. 9X of 1450 responding organizations were 
using computers for MIS applications at the time of the study, 
compared to 73% using computers for accounting and payroll 
applications. However, the report showed that 907o of the 
responding organizations planned to use computers for MIS 
applications in the future (p. 17). Thus, if there are major 
differences in the organizational and procedural variables 
related to success with MIS projects, as opposed to those 
variables related to success with non-MIS projects, findings of 
previous studies of the total organization's MIS efforts may 
not hold up in this new MIS environment. On the other hand, 
it may be that the same variables work in the same directions 
in both cases, or that the findings of earlier studies may have 
reflected the MIS part of the application mix. Unfortunately, 
the resolution of these uncertainties is impossible due to the 
nature of the data reported in previous studies. 

Finally, the research reported in this thesis is a 
break from the tradition of the intensive one-case study at 
one extreme, and the large-scale questionnaire survey at the 
other. The approach in this research has followed the lines 
proposed by Mouzelis (1969), in that a small sample has been 
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studied rather intensively; not as intensively as for a one- 
case study, but more intensively than is possible with a wide- 
spread questionnaire survey. Further, the sample, although 
small at twenty projects from ten organizations, represents 
a diversity of organizational environments, which affords the 
opportunity for greater generalization than the one-case approach. 

Organization of the Study 

The author's prime objective in conducting the 
research reported in this thesis has been to add to the body 
of knowledge concerning the development and implementation of 
MIS projects. Specifically, an effort has been made to 
determine what organizational and procedural factors relate to 
MIS project success. It was recognized at the outset that one 
research effort could only scratch the surface in a field where 
so little definitive research has been done. However, it was 
felt that a well structured exploratory study, dealing with 
empirical data, could provide not only answers to some perplex- 
ing questions in its own right, but, more importantly, help to 
define more explicitly those finer-grain questions upon which 
additional research could be focused. 

In pursuing the above objectives, the literature on 
information systems, computer data-process ing, and project 
management was reviewed to determine what other authors have 
had to say that was relevant to the subject at hand. The 



results of that review are reported in Chapter II. 
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Based upon the literature survey, the author's own 
experience in the infortnation systems field, and the suggestions 
of faculty members and fellow graduate students at the University 
of Minnesota, a research design was gradually evolved which it 
was believed would facilitate satisfying the research objectives. 
Essentially, the approach involved defining a number of hypotheses 
which could be tested with empirical data. These hypotheses, 
along with the criteria of MIS project success, are presented 
in Chapter III. 

Following the enumeration of hypotheses to be tested, 
and the specification of success criteria, the detailed 
methodology of the study was worked out. The original set of 
hypotheses was reduced to more manageable proportions; an 
interview instrument and procedures for data collection were 
developed and tested; the study sample was selected; and the 
method of data analysis was decided upon. Chapter IV deals with 
these aspects of the study methodology. 

The data that were collected have been presented in 
two ways: Chapter V deals with the organization and project 

samples in descriptive terms, which gives the reader a "feel" 
for the subjects included in the study; Chapters VI and VII 
present the results of statistical tests on the relationships 
among the hypothesis, criterion, and other variables for 
which data were collected. (See Appendix E for the list of 
all variables in the study.) 



-13 



Chapter VIII consists of a summary of the study 
findings, along with some interpretive comnents by the author 
Conclusions drawn by the author, as well as suggestions for 
future research directions, are presented in Chapter DC. 
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CHAPTER II 

SURVEY OF RELEVANT LITERATURE 

The survey of literature relevant to the procedural 
and organizational variables relating to success with MIS 
projects will be developed along two basic lines: that 

literature essentially prescriptive in nature but not founded 
on empirical studies, and that literature reporting the 
findings of empirical studies. 

Prescriptive Literature 

The prescriptive literature will be considered in 
two different categories. First to be dealt with will be the 
literature pertaining to what can generally be classified as 
organizational factors. This category includes such subjects 
as the location of the MIS function within the overall organi- 
zation structure, the degree of executive involvement in MIS 
development, the participation of user departments in MIS 
development, the selection and training of MIS staff, and the 
specific organizational structure for given projects,* 

The second category of prescriptive literature is 
concerned with MIS project planning and control factors, and 
is comprised of such topics as definition of project objectives, 
documentation, allocation of manpower resources to projects, 
cost estimating of projects, the development and application 
of standards to project personnel, and the use of various 
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pro Ject control techniques similar to CPM or PERT. 

Organizational Factors 

A topic that has received a great deal of attention 
in the literature on information systems is the location in the 
organization of the information systems function. Diebold (1964) 
has contended that organizations have failed to adapt to the 
requirements for effective information systems by not giving 
adequate attention to a specific place in the organization 
for MIS functions. Retchenbach and Tasso (1968) have pointed 
out that the location of the MIS function has been determined 
in many cases by the nature of early computer applications, 
not the requirements of the current situation. Since early 
use of the computer was often oriented to accounting functions, 
the computer activity was placed under the controller and has 
frequently been left there. Whitmore (1966) has argued that 
the location of the MIS function within the controller's 
organization is a natural placement, in consonance with the 
controller's overall responsibilities, and should, therefore, 
remain there. Others, notably Cole (1966) and Brandon (1970), 
have merely held that the MIS function should be placed at a 
high level in the organization, commensurate with the responsi- 
bilities of the MIS manager. A counter point of view to 
Whitmore's, expressed by Canning and Sisson (1967), and 
Aukerman (1966), is that the MIS function should definitely 
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not be under the controller or a financial executive; rather, 
that it should be located directly under the top executive as 
an independent function* Finally, Reif (1968) has reviewed 
the literature bearing on organizational placement of the MIS 
function and has provided an analysis of the different points 
of view. In suninary, it would appear that the literature on 
organizational location of the MIS function leans toward the 
prescription of independent function status at a high organi- 
zational level, although this position is not unanimously held. 

A second organizational factor which has received 
attention in the literature is the necessity for top management 
and user interest and participation in directing and controlling 
the development of MIS (Aukerman, 1966; Brandon, 1970; Canning 
6t Sisson, 1967; Gallagher, 1961). The basic position held in 
conmon among these writers is that only through active partici- 
pation and assumption of responsibility for results can execu- 
tives and managers expect to derive the potential benefits 
from their management information systems. In the absence of 
such positive direction, the MIS will end up being what the 
MIS staff provides, which may or may not be what is really 
needed (probably the latter). 

With respect to staffing the MIS function, Canning 
(1967) and Whitmore (1966), among others, have prescribed 
means of selecting, developing and organizing the MIS staff. 
Campise (1968) has paid special attention to the need for 
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highly qualified MIS project leaders who are capable of 
effective coninunication with user personnel. Finally, Canning 
and Sisson (1967), and Ditri and Wood (1969) have advocated 
the use of in ter functional project teams composed of MIS staff 
and user personnel as a means of achieving higher quality MIS 
products and user organization acceptance of change. 

Project Planning and Control Factors 

Various aspects of MIS project planning and control 
have been dealt with in the literature. Probably the best known 
sources are Lecht (1967) and Brandon (1963; 1968). Lecht covers 
virtually the entire range of activities relating to project 
planning and control, while Brandon's scope has been more 
narrowly confined to the development and application of perform- 
ance and methods standards* LaBolle, et al (1965), working at 
the System Development Corporation, have developed a comprehen- 
sive planning guide for computer project development which also 
deals with almost all areas of project planning and control. 

Schwartz (1964) has stated that ’’Inadequate planning 
is at the root of most data -processing failures and over-runs 
(p. 14).” He has proposed using a ”Work Breakdown Structure” 
to formalize and improve the planning, direction, and control 
of projects. Mensh (1969) has advocated using techniques which 
appear to be quite close to Schwartz's work breakdown structure. 

Blumenthal (1969) has provided what is probably the 
most thorough and articulate statement of the way in which an 
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organization should approach its total MIS effort, as well as 
how specific projects should be handled* Blumenthal advocates 
setting up an analytical framework based on functional areas 
of the organization. All proposals for MIS projects are then 
evaluated within the context of this framework. Blumenthal 
also provides a detailed, step by step procedure for the 
generation, selection, development, implementation, and audit 
of MIS projects. 

Some authors have recommended the use of such 
formalized project planning and control techniques as PERT 
(Canning & Sisson, 1967; Tannert, 1969) and PEST (Program 
Planning and Scheduling Technique, Anderson, 1966). The 
difficulty usually cited as inhibiting the utility of these 
critical path methods is the inability to make accurate 
resource requirement and time estimates. Pietrasanta (1968) 
has dealt explicitly with this problem in a discussion of 
estimates for manpower, machine- time, money, and project time. 

A much more thorough treatment of this same problem, and one 
based on empirical data collected by questionnaires for 169 
computer programs, has been provided by Nelson (1967). 

Finally, documentation methods and requirements for 
MIS projects have been proposed by Snyder (1965), Kay (1969), 
and Tannert (1969). In fact, with regard to project documenta- 
tion, nearly every writer in the field of MIS management has 
lamented the pervasive lack of adequate project documentation, 
and exhorted MIS management to recognize and deal with this problem. 
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Empirical Studies 

As was indicated earlier, all of the empirical 
research which has been reported (or at least found by the 
author) in the MIS area has focused on the overall organization 
rather than the individual project (with the exception of the 
SDC work on programming costs; Nelson, 1967). Most of this 
research has been accomplished either by management consultants 
or the American Management Association. The data for these 
various studies have generally been collected by questionnaire 
or interview, in some cases a combination of the two. For ease 
of presentation, the empirical studies will be discussed in 
chronolog ical order . 

Based on the experience of 124 companies with data 
processing systems, Baumes (1961) drew several conclusions which 
he put in the form of recommendations for any organization 
planning to develop a computerized information system. The key 
point made by Baumes was the need for detailed planning of the 
tasks involved in information systems development. Such planning 
should be predicated on a clear statement of organizational 
objectives, and should definitely give close attention to 
behavioral factors, as well as technical factors. 

Thurston (1962) has reported a study of 32 case 
histories of organizations* data processing activities. His 
primary conclusion was that a critical element in success of 
information systems was operating management's control of systems 
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planning. 

Perhaps the best known of the several studies which 
have been conducted by management consultants were the two by 
McKinsey & Company (Garrity, 1963; Unlocking the Computer's 
Profit Potential . 1968). Both of these surveys, based entirely 
on Interview data, were aimed at determining what organizational 
and methodological factors had the greatest bearing on the 
successful usage of computers in the organizations studied. The 
1963 survey consisted of a sample of 27 companies representing 
13 industries. The 1968 survey covered 36 companies, also 
representing 13 industries. Those generally interviewed were 
the chief executive officer, the head of data systems, and 
other selected members of management concerned with the problem 
of effective computer utilization.^ 

The conclusions drawn from both McKinsey surveys 
were essentially the same. These conclusions, briefly, were 
the need for top -management involvement from systems conception 
through implementation; the need for expecting top performance 
from the systems group across a wide spectrum of applications; 
and the need for the participation of user personnel in all 
levels of systems design and implementation. Those companies 
surveyed which were judged successful in their computer 

^This information was provided by Mr. David B. Hertz of 
McKinsey & Co, in personel correspondence with the author 
dated 26 November 1968. 
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operations all evidenced these organizational and managerial 
characteristics • 

A very general questionnaire survey of 300 business 
firms using computers was conducted in 1964 by Business Automation 
(Kornblum, 1964)* The objective of this survey was to find out 
how successfully computer installations were being used and 
managed. It appeared that the questionnaires were completed 
mainly by data processing personnel as the conclusions seemed 
to reflect the attitudes of the computer systems staff toward 
others in the organization. For the most part, the opinions 
expressed indicated that those outside the computer systems 
function did not understand the computer or how to use it; they 
did not understand the unique problems and needs of computer 
systems personnel; and they resented the encroachments of computer 
systems in their functional areas. 

The American Management Association sponsored a 
survey in 1965 (Higginson, 1965) with the primary objective 
of determining how the companies studied were using computer 
equipment, and the impact of computer systems on their organiza- 
tions and operations. This survey was conducted both by question- 
naire and interview. Questionnaires were returned by 288 
companies of 966 solicited, and interviews were conducted in 
21 firms to get more in-depth information as well as to get a 
subjective ”feel** for the attitudes of management personnel. 



The conclusions were quite similar to those already cited from 
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earlier studies: the need for greater top -management direction 

and control of the computer systems function; the need for 
closer integration of staff specialists and operational manage- 
ment in the development and evaluation of information systems; 
and the need to consider the computer systems function just like 
any other functional unit with respect to organizational 
structure and evaluation of effectiveness in terms of corporate 
objectives . 

Two comprehensive surveys have been conducted by 
Booz, Allen and Hamilton, Inc., the first in 1966, and the second 
a year later ( Computer Operations in Manufacturing Companies . 
1966; Computer Management in Manufacturing Companies . 1967). 

Both surveys were virtually identical in purpose, varying 
primarily in the sample sizes and means of collecting data. 

In the first Booz, Allen 6e Hamilton survey, the 
stated purpose was **to determine what successful manufacturing 
companies have done, are doing, and plan to accomplish in 
using and managing the computer.” The key here was ’’successful”. 
Only those companies in the various industries studied with the 
fastest growth and greatest return on investment were surveyed, 
since it was the objective to determine how exceptionally well 
managed companies were using the computer. The second survey, 
conducted in 1967, was different only to the extent that a wider 
range of data covering an expanded sample was sought. 






i 













I 


















•# 



-23 



The 1966 survey was conducted entirely by interview, 
using no questionnaire as far as can be ascertained. Thirty- 
three companies were included in the survey, and 189 interviews 
were conducted within them. Over half of the in-depth inter- 
views were conducted with top and operating management. The 
findings were then evaluated in an attempt to correlate computer 
usage and management with success of the company in question. 

The 1967 survey used a sample of 108 manufacturing companies, 
and questionnaires were employed to collect the data. The 
questionnaires covered organizational structure and practices 
in each company, as well as specific information on computer 
costs, usage and plans. 

The conclusions drawn from both surveys were mainly 
enumerations of the various composite computer management 
characteristics for successful companies. The reports did 
indicate that a trend was developing toward more top management 
and operational management involvement in computer systems plans 
and audits. Further, it appeared that the trend was to move 
the overall responsibility for computer activities away from 
financial executives to a computer executive at top corporate 
level. 

A questionnaire survey, conducted in 1967 as part of 
the Diebold Research Program ( Summary Report of a Survey on 
the Cost Effectiveness of Software & Hardware . 1967), was 



designed to explore the future management implications of 
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computer information systems, and to develop some standard 
measurements of the cost effectiveness of hardware and software. 
The respondents to the questionnaire represented 17% of those 
solicited, and consisted of senior information systems manage- 
ment, technical staff reporting to information systems manage- 
ment, and top corporate management. The findings of relevance 
here were that in the majority of cases top management was not 
active in guiding the growth of information systems activities, 
and that new computer applications were often proposed by 
information systems management rather than those closer to the 
application area, namely, user management. The major problem 
with proper utilization of computer information systems was 
found to be the lack of concern on the part of management for 
maximum utilization. It was also concluded that larger computer 
users were more likely to have budget and control procedures 
for information systems activities, and were generally getting 
greater return on their information systems investments than 
smaller users. 

Lipperman (1968) has reported the results of investi- 
gations in twelve organizations using computer information 
systems. His data, collected by interview, were organized on 
a case basis to portray what various types of organizations 
have done to successfully implement computer systems. Lipperman 
dealt with both ‘‘operative” systems and planning systems, and 
pointed to the latter as the direction in which information 



- 25 - 



systems should develop as operative subsystems are integrated 
at higher levels* This particular study would seem to be the 
first dealing specifically with MIS as defined in this thesis. 

Several points were made by Lipperman which are of 
special interest here. As organizations moved toward higher 
level systems that integrated information from various subsystems, 
more user management responsibility for defining system require- 
ments was essential. Following from this was the need to 
locate the information systems function high in the management 
hierarchy, independent of such other functional executives as 
the controller or financial vice-president. 

A study undertaken for the American Management Associa- 
tion, under the direction of Reichenbach and Tasso (1968), 
was based on the hypothesis that *'the location of the computer 
complex within the organization will affect the complex*s opera- 
tions (p. 16).*' This study consisted of in-depth interviews 
with three levels of managemeifet in sixteen companies representing 
nine industries. The organizational members interviewed were 
top executives, information systems personnel, and user organi- 
zation managers. Although numerous conclusions were drawn 
from the sixteen case studies, the gist of the findings can be 
summarized briefly as follows: 

The location of the computer complex does have a 
very real relationship with its effectiveness in 
meeting organization needs. (No evidence was 
provided on the measure or meaning of effectiveness.) 
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The computer information systems function 
should report directly to top management, 
and should not be a part of any existing 
trad i t iona 1 f unc t ion . 

The centralization of the computer activi- 
ties need not have any impact on the 
traditional form of the corporate structure. 

The latter is a function of management's 
philosophy and objectives. 

The nature of an organization's basic 
activities and processes bears on the 
ease of its development of computer infor- 
mation systems. 

Participation by top level and user manage- 
ment in the development of information 
systems will enhance acceptance of those 
systems, improve utilization of the compu- 
ter, and provide results more responsive 
to management's needs. 

A monograph by Reif (1968) reported the findings of 
research he conducted for his doctoral dissertation in 1966. 

The literature survey comprising Chapter 2 of this monograph 
has been cited previously. The focus here will be on Reif's 
research methodology and findings. The impetus behind Reif's 
research was his belief that a major obstacle facing organiza- 
tions attempting to achieve maximum benefits from computer 
systems was the conflicts between the imperatives of computer 
technology and the traditional structure of business organizations. 

**There is a growing number of we 11 -documented 
studies which show that the two are not com- 
patible; yet most firms today are content to 
achieve limited results by super-imposing 
advanced information systems on organizational 
structures which are unable to adjust to the 
new requirements and interpersonal relation- 
ships dictated by the systems design (from 
the Preface, p. iv).** 
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The specific purpose of Reif*s study was '*to examine what 
structural changes occur following the implementation of 
computer systems in business organizations (p, 41).** This 
purpose was enumerated into several explicit factors such as 
the locus of decision making within the management hierarchy, 
formal and informal communication channels, and the functional 
integration of organizational activities. In addition, Reif 
studied the role of the computer systems staff, its location 
in the organization, and its relationships with user departments. 

Reif used the case method for studying three firms: 
a utility company, a bank, and a manufacturing firm. Most of 
the data were collected by interviews, both structured and 
unstructured. The remainder of the data came from company 
records. Before considering Reif's findings, it is important 
to note that his data reflect a considerable number of non-MIS 
computer applications, what might be termed **produc tion** jobs, 
as opposed to the kind of decision oriented management informa- 
tion which is the focus of the current research. 

Based on his research, some of Reif*s conclusions 



were: 



The presence of the computer resulted in 
centralizing decision-making within the 
management hierarchy. 

Line-staff relationships were changed. 
The computer staff group exerted great 
influence and made decisions affecting 
the internal activities of operating 
departments. This staff influence was 
de facto, not usually legitimized by 
formal organization structure. 
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Lateral flows of communication throughout 
the management hierarchy were formed and 
expanded. However, formal channels of 
communication were not generally revised. 

The impact of computer systems will be 
greatest on middle management as routine 
administration and control decisions 
become programmed; these positions will 
generally decrease in number. However, 
the impact on middle managers will vary, 
depending on the organization, top 
management’s objectives, and the 
functional area concerned. 

The hierarchical structure will be 
retained in organizations but ’’certain 
changes in the management framework 
appear to be imperative If the firm 
Is to maintain harmony between infor- 
mation technology and the structure 
of organization (p. 114).” 

In an attempt to measure the effectiveness of manage- 
ment Infoirmatlon systems in terms of profitability, a questionnaire 
survey was initiated by S. D. Leldesdorf & Co. in 1967 ( The 
Profitability of Management Information Sys terns > 1969). Of 142 
companies requested to participate in this survey, 130 did so. 

These 130 companies, representing eleven Industry groupings, 
were known to be using computers, ’’presumably effectively.” 

Although the objective of the Leldesdorf study dealt 
specifically with the terms MIS and profitability, the use of 
these terms may be somewhat misleading. The profitability 
measure was derived by asking the participating companies ”to 
express as a percentage the influence of their computer operations 
and associated expenses on profitability during their first five 
years (or less, where that was appropriate) of computer-based 
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data processing (p. 18)/* This approach raises two issues: 
first, from information contained in the study report, it is 
impossible to determine what data went into each company's 
response on profitability, or how that data was aggregated 
to make the requested computation; and second, since the 
requested computation dealt with the early period of each 
company's computer use, it is highly probable that return from 
MIS applications was greatly confounded, if not overshadowed, 
by return from non-MIS applications. This latter observation 
is made in view of the tendency for organizations installing 
computers to implement cost-displacement applications first, 
with MIS applications following as more experience and sophis- 
tication are acquired. 

With respect to the results of the Leidesdorf study, 

32 of the 130 companies were viewed as being successful with 

their MIS in terms of the reported percentage profitability 

contribution of MIS. The study concluded that these 32 

companies achieved the high MIS contribution to profitability 

by ''intelligent planning and control**. For the 98 companies 

viewed as not successful in terras of MIS contribution to 

profitability, the report stated that: 

"The common denominator seems to be defi- 
ciencies in planning associated with the 
operation. To a lesser extent, the absence 
or incompetence of functional and/or execu- 
tive participation in the planning and 
operation of the project appears to be a 
contributing factor (p. 21).'* 
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To summarize the Leidesdorf study, it was concluded 

that: 

“Management information systems represent 
a favorable field for investment and can 
make a real contribution to prof itability-- 
especially through improved control of the 
business; but positive results are not 
automatic and guaranteed. Serious problems 
can and do arise and are usually attribut- 
able to deficiencies in management involve- 
ment, planning, and control (p. 24), '* 

Before concluding this section on the survey of the 
literature, two doctoral dissertations warrant mention. Holsinger 
(1970) studied the constraints to implementation of data process- 
ing systems, and examined the roles of key decision makers 
involved in implementing such systems. Holsinger obtained his 
data through interviews with members of top management, key 
technical personnel, and “third party representatives," The 
constraining variables were categorized as technological, organi- 
zational, and individual. Holsinger concluded that the relative 
importance of these three categories has shifted over time; 
that as size increases, the importance of technology and organi- 
zation increases at the same time that the importance of the 
individual decreases; that size influences the organization's 
ability to utilize a proper division of labor; that control 
problems were created through the determination by technologists 
of critical aspects of systems design; and that structural 
problems of integrating systems functions with various types of 
information systems appeared. 
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The second dissertation, by Hcdgetts (1968), is of 
interest since it deals with project management. Based on the 
research he conducted, Hodgetts advocates the project approach 
as opposed to the functional approach where an organization is 
confronted with managing a specific, complex task with stringent 
quality parameters. He enumerates the pros and cons of the 
project management approach and points out that it is a manage- 
ment tool not an organizational panacea. Of particular interest 
are his observations concerning the interface problems between 
the project organization and the permanent organization, and 
the attributes of effective project managers. Although Hodgetts 
was concerned primarily with project management in the aero- 
space, construction, chemical, and consumer products industries, 
and not with MIS projects, his inter-industry approach, the 
focus on individual projects and the evaluation of project tech- 
niques and relationships are relevant to the present research. 

Summary 

Although the literature relevant to management infor- 
mation systems is rather extensive, there are few reports of 
well conceived and structured research in the field. While 
prescriptions and definitions have been plentiful, they have 
seldom been supported by data. Numerous authors have presented 
their views on virtually all aspects of information systems, 
but these views have generally stemmed from their authors’ 
personal experiences rather than structured research. Those 
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research studies which have been reported have tended to be 
broad-brush. They have generally represented wide-scale 
questionnaire and interview surveys which were aimed at overall 
organizational use of computers, or case studies of the impact 
of computer usage on organization structures and relationships. 

The above comments concerning the literature relevant 
to MIS are not intended as criticism of what others have done. 
Rather, the point to be made is that in a field growing so 
rapidly, with so few definitive guidelines, a great deal more 
needs to be done. Researchers and practitioners in the MIS 
field must define areas to be researched, then develop well- 
conceived approaches to conducting such research. What has 
already been accomplished will be helpful, as, indeed, it has 
been helpful to the author in conceiving and designing the 
research reported in this thesis. But the overwhelming impression 
one is left with after reviewing the information systems litera- 
ture is the great gap between what is needed and what has been 
accomplished. 

Building on what others have said and done, the 
author has made an attempt to start closing that gap. The 
prescriptions and empirical findings of others were major 
contributions to the definition of the hypotheses posited and 
tested in the present research. The techniques others have 
employed have been invaluable guides to the author in designing 
the data collection methodology used in this research. 
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Nonetheless, the research conducted by the author, and reported 
in this thesis, represents a departure from previous research 
in both approach and substance. Rather than collecting a mass 
of data, then setting up hypotheses to be tested, the author 
of the present study set up the hypotheses, then developed data 
collection and analysis techniques to test them. As opposed 
to looking at overall computer usage in many organizations, or 
the intensive study of one or two organizations to determine 
what impact computer usage has had on them, the present research 
has focused on the MIS project. Specific, measurable criteria 
of project success have been defined in order that the hypotheses 
posited could be tested. 
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PART II 



HYPOTHESES, CRITERIA OF SUCCESS, AND 
METHODOLOGY OF THE STUDY 



This part of the thesis deals with the means by which it was 

attempted to formulate answers to the question posed earlier 

**What organizational and procedural factors 
are correlates of success with MIS projects?*’ 
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CHAPTER III 

HYPOTHESES AND CRITERIA OF SUCCESS 
Introduc tion 

To assist in identifying the organizational and 
procedural factors related to success with MIS projects, a 
list of thirty-five hypotheses was developed. These hypotheses 
were constructed from the literature previously reviewed, from 
the author's personal experience, and from discussions with 
various professionals in the MIS field. The hypotheses provided 
a framework for the collection and analysis of data relevant 
to the purpose stated above. 

In addition to the list of hypotheses, a set of 
criteria was developed to evaluate relative success of given 
MIS projects. The criteria will be taken up first, followed 
by a discussion of the hypotheses. 

Criteria of Success 

A given MIS project can be viewed from several 
perspectives in attempting to Judge whether or not it was 
successful. It is necessary that the criteria selected be 
relevant, be as objective as possible, and at the same time be 
realistic in terms of ease and consistency of measurement. Four 
separate criteria which appear to meet these requirements are: 

1. Success in terms of time. How closely 
did actual time spent on the project 



conform to the time allocated for it? 
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2* Success in terms of development cost. 

Was the project completed within the 
budgeted cost? If over-runs were 
experienced, what was the total cost 
as a percentage of budgeted cost? 

3. Success in terms of users* evaluations. 

Did the completed project satisfy user 
requirements and expectations? Were 
there unanticipated problems or benefits 
resulting from implementing the completed 
project? 

4. Success in terms of computer operations. 

Has the completed project created problems 
for computer operations in scheduling, set 
up, run time, or control? 

It is believed that the above criteria are relevant 
operational measures of project success, and, as such, provided 
a suitable means of testing the hypotheses selected for evaluation. 

Hypotheses 

The thirty-five factors which were hypothesized to be 
related to MIS project success fall into the following broad 
categories : 

1. Characteristics and competence of project 
personnel . 



2. Project management and control techniques. 
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3. Organizational Interaction factors. 

4. Project specific factors. 

5. Global factors comprising the project 
environment. 

The hypotheses were not viewed as exhaustive nor were their 
assignments to categories clear cut in all cases. However, it 
was felt that both the hypotheses, and the categories into which 
they were placed, provided a suitable framework for conducting 
research. 

The hypotheses are presented below by categories, 
with a brief description of what each category represents. The 
individual factors or hypotheses should be self-explanatory. 

In order to avoid redundancy in the list of hypotheses, an 
example of the form for each full hypothesis will be given 
here. The list itself will merely consist of the body of the 
given hypothesis in the form of a factor ostensibly related to 
MIS project success. 

Example; It is hypothesized that the higher 
the level of formal education of 
project personnel, the greater the 
likelihood of MIS project success. 

Characteristics and Competence of Project Personnel (Category I) 

This category should require little explanation. It 
is merely a grouping of those hypotheses about the competence 
of people who worked directly on the project. While the general 
statement '*the more competent the personnel working on the project. 
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the more likely success’* is something of a truism, the factors 
falling under competence may be debatable in certain cases. 
Such factors are, then, testable. 

1. Coordinating ability of project leader. 

2. Systems experience of project personnel. 

3. Persuasiveness of project leader (as 
evaluated by superiors). 

4. Proficiency of project personnel (as 
judged by superiors). 

5. Low turnover of project personnel. 

6. Length of experience in the organiza- 
tion of project personnel. 

7. High formal educational level of 
project personnel. 

Organizational Interaction Factors (Category II) 

This category includes those hypotheses dealing with 
how the user organization interfaces with, supports, and is 
supported by the MIS department. It is, in short, a group of 
hypotheses focused on the integration of the overall organiza- 
tion with respect to development and utilization of management 
information systems. 

8. Participation by operating management in 
design, formal approval of specifications, 
and continual review of project. 
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9. Utilization of a project team composed 
of MIS staff and user personnel. 

10. Operating management conducts periodic 
management audit of MIS function. 

(Evaluation of effectiveness for users). 

11. Organizational level of top computer 
executive. 

12. Formal training program set up for 
user organization. 

13. Project originated by user organization. 

Project Management and Control Techniques (Category III) 

Subsumed within this category are those hypotheses 
pertaining to specific methods used to plan, direct, monitor, 
and control a given project. This group represents many of 
what are frequently prescribed as *'good practices*' in the 
literature on management of the data processing function. 

14. Measurable project objectives from 
conception of the project. 

15. Formal project selection process used 
to determine which projects to develop. 

16. Documentation standards used and enforced. 

17. Use of a formalized and regular reporting 
structure on project progress. 

18. Planning and accounting for all resources 
throughout project development. 
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19. Program maintenance and review responsibility 
specified for definite period after implementation. 

20. Utilization of a formal time-scheduling 
technique such as PERT for project development. 

21. Performance standards employed for analysts 
and programmers. 

Project Specific Factors (Category IV) 

This category encompasses the various factors that 
are unique to a given MIS project, such as the programning 
language used. 

22. High availability of computer time for 
program testing. 

23. High level programning language used 
for project. 

24. Utilize existing data base versus 
constructing or greatly modifying one. 

25. Short-term, minor project versus large, 
complex project. 

Global Factors Making up the Project Environment (Category V) 

The global category covers a rather wide range of 
factors as the name would imply. These are environmental 
variables which might be viewed as more indirect in effect than 
the factors under other classifications. In a sense this is a 



sort of omnibus category that includes the hypotheses that were 
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cons idered relevant but did not easily fit into a more specific 
grouping. 

26. High centralization of organizational 
MIS activities. 

27. Number of years experience for organiza- 
tion with computerized information systems. 

28. Low turnover rate of MIS staff. 

29. Combination analyst /programmers for 
small projects. 

30. High rates of MIS staff drawn from 
within the organization. 

31. High average income level of MIS staff. 

32. Low degree of overall organizational 
change. 

33. Separation of analysts and programmers 
for large projects. 

34. Overall size of organization systems staff. 

35. Ratio of computer hardware investment to 
total sales or operating budget. 

Summary 

This chapter has been devoted to presenting the factors 
which were hypothesized to be related to MIS project success and 
to de fining specif ic criteria of success suitable for testing 
those hypotheses. Chapter IV details the methods which were used 



to test the hypotheses. 
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CHAPTER IV 

METHODOLOGY OF THE STUDY 

Initial Evaluation of the Hypotheses 
The above list of thirty-five hypotheses would Impose 
a considerable burden on anyone desiring to test them In field 
research. For this reason, It was felt advisable to try to 
sort out the various hypotheses Into some kind of priority 
hierarchy. The result of such a sorting process would be a 
ranking of all the factors, from those viewed as most crucial 
to project success to those viewed as least significant. It 
was decided that the best means of coming up with such a 
priority ranking of the hypotheses was to get the opinions of 
professionals In the MIS field. 

The opportunity for getting the opinions of MIS 
professionals on the Importance of the various hypotheses to 
project success was afforded by the Founding Conference of the 
Society for Management Information Systems (SMIS), held at the 
University of Minnesota In late 1969, The decision was made to 
submit the hypotheses In questionnaire form to the attendees of 
this conference. The expected product of the completed question- 
naires was to be the desired priority ranking, 

Pre-ques t lonna Ire 

To facilitate developing a questionnaire which, when 
completed, could be evaluated In such a way as to provide the 
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ranking by importance of the hypotheses, it was decided that a 
pre-questionnaire should be prepared and submitted to a group 
of MIS professionals in the Twin Cities area. The respondents 
(N=43) for the pre-questionnaire were all employed by the firms 
associated with the Management Information Systems Research 
Center of the University of Minnesota. 

The objective in preparing and evaluating the pre- 
questionnaire was to come up with a "neutral** hypothesis which 
could be used as a reference or anchor point in constructing the 
final questionnaire to be submitted to the SMIS conference 
participants. This neutral hypothesis was to be the basis of 
comparison for each of the remaining thirty-four hypotheses. 

By choosing a "neutral" hypothesis as the standard for comparison, 
one seldom viewed as either most Important or least important 
to MIS project success by the pre-questionnaire respondents, 
it was expected that the desired spread in priority rankings 
would be derived from the final questionnaire. 

The procedure employed for the pre-questionnaire was 
to list the thirty-five hypotheses, or factors, disregarding 
any classification by categories. Opposite each hypothesis 
were two blanks, one under a column headed '*Most Important" 
and the other under a column headed *'Least Important*'. The 
respondent was then requested to evaluate each of the thirty- 
five factors as to its criticality for MIS project success, 
and to indicate by the numbers 1 to 7 those seven factors 



- 44 - 



consldered to be most important, and those seven factors 
considered least important, in contributing to project success. 
(See Figure 1). 



FIGURE 1 

Pre-ques tionnaire 



Factor 



Most Least 

Important Important 



1) High formal educational level 
of project personnel 

2) Proficiency of project personnel 
as judged by superiors 



35) Etc. 



E tc . E tc . 



The returned pre-questionnaires were analyzed to 
determine which factor was "most neutral". That is, which 
factor was regarded to be of middle importance in the range of 
thirty-five. This factor was, of course, one that was seldom 
ranked as one of the seven most or seven least important in 
contributing to MIS project success. 

SMIS Questionnaire 

Analysis of the responses showed the "most neutral" 
factor to be: "Performance standards employed for analysts 



and programmers". This factor was then used to construct the 
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final questionnaire in the following manner. The remaining 
thirty-four factors were listed as before with no categorization. 
Opposite each factor was a Likert-type scale allowing for one 
of five responses to each item. At the top of each page of the 
questionnaire the neutral (standard) factor was printed. The 
respondent was then asked to rate each of the thirty-four 
remaining factors for importance in contributing to MIS project 
success by comparing it to the standard factor. The form of 
the response was merely a check in the chosen one of the five 
blanks. In addition, each respondent was provided space at the 
end of the questionnaire to write in any other factors which 
he believed important to MIS project success, but which were 
not included in the list of thirty-four factors to be rated 
(see Appendix A). 

Of the roughly 250 participants in the SMIS conference, 

142 returned the questionnaire. The returned questionnaires 

were tallied by determining the number of responses in each of 

the five levels for each factor. Each level was then given a 

numerical value as follows: 

Much less 1 

Somewhat less 2 

About same 3 

Somewhat more 4 

Much more 5 

Multiplying the number of responses per level by the 

level value, summing across all levels for a factor, then 
dividing by the number of responses for that factor, yielded a 
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numerical score for each factor. The factors were then ordered 
on the basis of these scores to give a ranking from most 
important to least important. 

Results from the SMIS Questionnaire 
Evaluation of the returned questionnaires resulted in 
the factor ranking shown in Figure 2. The scores for the 
individual factors ranged from 4.38 for the factor judged most 
important for MIS project success to 2.30 for the least important 
factor. In addition, as will be noted in Figure 2, the variance 
was calculated for each factor. These variances, ranging from 
.602 for the factor ranked second in importance to 1.519 for 
the factor ranked seventeenth, were computed to determine the 
relative agreement among respondents on the factors, and were 
not used for any purpose beyond that. 

Analysis of SMIS Questionnaire Data 

As was pointed out earlier, the questionnaire was 
constructed and administered to facilitate field study of the 
hypotheses. The ranking provided a means of focusing on what 
factors those active in the MIS field believed to be the most, 
the least, and of intermediate importance to project success. 

The next steps were to select the hypotheses to be further 
evaluated, and to design a means of collecting data in organi- 
zations on both the hypotheses and the criteria. 



■47 



FIGURE 2 

Ranking of Hypotheses 



Rank 




Computed 

Score 


Variance 


* 1 


Participation by operating management 
in design, formal approval of specifi- 
cations and continual review of 
project. 


4.38 


.676 


* 2 


Measurable project objectives from 
conception of the project. 


4.29 


.602 


* 3 


Utilization of a project team composed 
of MIS staff and user personnel. 


4.25 


.643 


4 


Coordinating ability of project leader. 


4.21 


.620 


5 


Operating management conducts periodic 
management audit of MIS function. 
(Evaluation of effectiveness for users) 


. 4.00 


.970 


6 


Formal training program set up for 
user organization. 


3.95 


.872 


* 7 


Organizational level of top computer 
executive. 


3.94 


1.004 


* 8 


Systems experience of project personnel 


. 3.90 


.642 


9 


Formal project selection process used 
to determine which projects to develop. 


3.88 


1.058 


10 


Persuasiveness of project leader 
(superior's evaluation). 


3.84 


.856 


11 


Proficiency of project personnel 
(as judged by superiors). 


3.79 


.780 


*12 


Documentation standards used and 
enforced. 


3.74 


.942 


*13 


Use of a formalized and regular report- 
ing structure on project progress. 


3.68 


.686 


*14 


Low turnover of proiect personnel. 


3.56 


.862 



* Hypotheses selected for further study 
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Rank 

15 

*16 

*17 

18 

19 

*20 

21 

22 

*23 

24 

25 

26 

27 



FIGURE 2 (Cent.) 



Computed 

Score 



Planning and accounting for all 
resources throughout project develop- 
ment. 3.50 

Source of origination of project (MIS 

staff or user). 3.48 

High centralization or organizational 

MIS activities. 3.46 

Program maintenance and review 

responsibility specified for definite 

period after Implementation. 3.45 

Nvnnber of years experience for 
organization with computerized 
Information systems. 3.43 

Length of experience In the organiza- 
tion of project personnel. 3.32 

Utilization of a formal time-scheduling 
technique such as PERT for project 
development. 3.25 

High availability of computer time for 
program testing. 3.04 

High level programming language used 

for project. 3.00 

Utilize existing data base versus 
constructing or greatly modifying one. 2.99 

Low turnover rate of MIS staff. 2.91 

Short-term, minor project versus 

large, complex project. 2.89 

Combination analyst/programner for 

small projects. 2.86 



Variance 

.968 

1.237 

1.519 

1.073 

.934 

1.051 

1.056 

1.272 

1.059 

1.511 

1.029 

1.437 

1.191 



* Hypotheses selected for further study 
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FIGURE 2 (Cont.) 



Computed 



Rank 




Score 


Variance 


28 


High rates of MIS staff drawn from 
within the organization. 


2.81 


1.104 


29 


High average Income level of MIS staff. 


2.77 


.750 


30 


Low degree of overall organizational 
change. 


2.72 


1.263 


★31 


High formal educational level of 
project personnel. 


2.71 


1.109 


*32 


Separation of analysts and programmers 
for large projects. 


2.51 


1.274 


*33 


Overall size of organization systems 
staff. 


2,50 


1.207 


*34 


Ratio of computer hardware Investment 
to total sales or operating budget. 


2.30 


1.188 



* Hypotheses selected for further study. 
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Two approaches were used in selecting the hypotheses 
for further evaluation. First, it was desired that about three 
factors be evaluated from the top, middle, and bottom of the 
ranked listing. Based on this, and the relative ease of 
collecting data on each of the factors, the following factors 
were tentatively selected for evaluation (numbers refer to 
ranks in Figure 2): 



Top 1, 2, 3 

Middle 14, 16, 17 

Bottom 31, 32, 33, 34 

Second, a factor analysis was performed on the 
questionnaire data using the principal components method 
(see Guilford, 1954; Harman, 1965). The reason for this was 
to determine if items on the questionnaire represented certain 
basic dimensions or underlying factors. If present, such 
underlying factors could then be used as a guide in selecting 
individual hypotheses from the ranked list for evaluation. 

The factor analysis yielded eleven factors 
accounting for 65.97« of the variance. However, most of these 
factors accounted for a small part of the overall variance, 
and/or only one or two questionnaire items loaded to any 
appreciable extent on a given factor. The factor accounting 
for the largest amount of the variance (9.4757,) represented 
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essentially those items that were ranked high on the question- 
naire (items 1, 2, 3, 5, 6, and 9 in Figure 2), The factor 
accounting for the second largest amount of the variance 
(9,4567«) represented essentially those questionnaire items 
ranked low (items 17, 28, 29, 31, 32, 33 and 34 in Figure 2), 

A third factor, accounting for 6.447o of the variance, had 
three items loading very high on it (items 13, 15 and 21 in 
Figure 2), and represented what could be called "project 
control". The other eight factors were judged not to be 
meaningful for the purposes here, either because they accounted 
for such a small part of the variance or because only one 
or two items loaded on them to any extent. 

It was apparent from the factor analysis that the 
high ranked and low ranked items, comprising the first two 
factors, were well represented in the preliminary list of 
ten hypotheses selected for further evaluation. However, 
the project control factor was not represented among those 
ten. Therefore, item 13 from the ranked list (Figure 2) was 
added to the ten hypotheses previously selected for further 
evaluation. 

Besides the above eleven hypotheses, five more 
were selected to be further evaluated. These were items 7, 

8, 12, 20, and 21 from the ranked list of hypotheses (Figure 2). 
The rationale for including these additional hypotheses was 
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(1) ease of collecting data concerning them; merely being 
in an organization would allow getting data on the hypotheses 
at virtually no extra cost, and (2) personal interest in the 
hypotheses because of related research currently going on 
or because of emphasis in the literature. 

As was noted earlier, space was provided for 
each respondent to write in any factors he felt important 
to MIS project success. All write-in responses were reviewed, 
but no new hypothesis was deemed necessary from this review. 

In most cases, the respondents merely restated in a different 
form one of the hypotheses in the list rated. In a few cases 
the write-in comments were not closely related to the 
hypotheses rated; however, the incidence of one or two comments 
on a subject was not viewed as justification for creating a 
new hypothesis. 

In summary, the result of the questionnaire 
analysis was the selection of those sixteen hypotheses identi- 
fied by an asterisk (*) in Figure 2 to be evaluated by field 
s tudy . 



Data Collection in the Field 
It was believed that the best means of collecting 
data to test the selected hypotheses was through interviewing 
key persons involved in developing, or using the products of. 
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MIS projects in several organizations in several industries. 

This approach was to comprise a ’'descriptive" study, using the 
terms of Selltiz et al (1967). That is, the emphasis was on 
finding out if given variables were associated, in this case the 
hypotheses and the criteria of project success. 

With a view to the above objectives, a questionnaire 
to be used by the researcher in structured interviews was 
developed. This questionnaire (see Appendix B) was aimed at 
collecting several kinds of data and consisted of several parts. 
First, the kinds of data collected with the questionnaire can 
be described as independent variable data, dependent variable 
data, and moderator variable data. The independent variables 
were the hypotheses selected, as outlined above; the dependent 
variables were the criteria discussed earlier; and the moderator 
variables represented various organizational or procedural 
factors which might have influenced or moderated the relation- 
ships among the independent and dependent variables. 

The parts of the questionnaire reflect who was expected 
to answer certain kinds of questions about a project in the 
organization. Thus, certain questions about the organization 
itself were to be answered by the manager of the MIS function. 

The questions specifically related to the project were to be 
asked of the project leader, and the questions concerning user 
perceptions of participation and satisfaction with the products 
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of the project were to be directed to user management 
personnel. ^ 

It was decided that twenty MIS projects, drawn from 
ten separate organizations, would constitute an adequate 
sample for the type of research to be conducted. This would 
represent two projects in each of the ten organizations. It 
was further decided that of the two projects in each organiza- 
tion, one of them should be viewed as relatively successful, 
and one relatively unsuccessful, by personnel in the informa- 
tion systems area of the organization. 

The general guidelines for selecting projects for 
study were specified as follows: 

1. Must be a MIS project as defined in Chapter I; 
not a data processing application with no 

MIS implications. 

2. At least two people worked full time in 
developing the project; the project required 
at least three elapsed months to develop. 

3. The project was completed and implemented 
within the last 6-24 months. 

The reason for guideline 2 above was to preclude studying 
very small projects which were completed by one person over a 
brief time, such as a simple report generation application from 

^The following references were used in developing the 
questionnaire for conducting interviews: Cannell & Kahn, 

1953; Edwards & Kenney, 1946; Festinger & Katz, 1953; Payne, 
1951; Selltiz et al, 1967; Torgerson, 1953; Whitlock, 1963. 
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an existing file of data. The reasons for guideline 3 were 
to avoid projects which had been completed for so long that 
those involved were not available for interview, or, if 
available, their recollections were too fuzzy to be anywhere 
near accurate; and to avoid projects which had been completed 
so recently that there was inadequate user experience with 
the products to allow fair evaluation. 

Pretest of Interview Instrument 

It was next decided that a pretest of the questionnaire 
should be made prior to actually selecting a sample and making 
contacts for the full study. The objectives of the pretest 
were to determine if data of the types wanted could be gathered, 
and, if so, were the questionnaire and associated structured 
interview effective and efficient means to collect these data. 

Accordingly, to satisfy these objectives, and to 
obtain estimates of how long the data collection would take 
for each organization in the sample, a pretest of the question- 
naire and interview technique was conducted during the week of 
4 May 1970, in a Twin Cities financial institution. Based on 
the pretest, it was concluded that the data collection techniques 
planned were feasible, and would provide the data desired. 
However, certain very minor changes to the questionnaire appeared 
desirable as a result of the pretest experience. For the most 
part, the changes merely involved revisions of wording in 
questions which had caused some confusion among pretest 
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respondents. Two questions were dropped completely from the 
questionnaire as It was apparent they were redundant. These 
two questions elicited essentially the same Information from 
respondents as two other questions which were left In the 
questionnaire. Three questions were added to the questionnaire, 
and two were modified substantially. In order that project 
leaders and users would be asked Identical questions concerning 
user participation and Implementation problems. Hone of these 
changes altered the substance of the questionnaire to any 
appreciable extent, and were, primarily, corrections for over- 
sights brought out by the pretest. After revising the question- 
naire, work began on selecting the organization sample to be 
used In the full field study. 

Organization Sample Selection 

As was stated at an earlier point. It was desired 
that the ten organizations comprising the study sample represent 
different Industries to the greatest extent possible. This 
objective for selecting an organization sample was constrained 
by the fact that It was also desired that those organizations 
In the sample be Associates of the Management Information Systems 
Research Center (MISRC) at the University of Minnesota. 

However, this constraint was not a very serious one In that the 
Associate firms do represent a wide cross-section of Industry 
types In the Twin Cities area. 
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With a view to the above requirements, a preliminary 
list of ten firms was prepared. Each of these ten firms was 
then sent a letter from the Associate Director of the MISRC, 
indicating the nature of the research to be conducted, the 
background of the researcher, and a request that the firm 
participate in the research (Appendix C). Within one to 
two weeks from the time the letter went out, the researcher 
contacted the Associate representative of each firm by tele- 
phone to request an appointment for the purpose of discussing 
the study and getting agreement from the firm to participate. 

During the last two weeks of July, 1970, each of 
the ten firms originally selected was visited by the researcher. 
These visits generally lasted about thirty minutes, during 
which time the researcher described what the study was to 
consist of, the kinds of data to be collected, and the types 
of people in the organization who would be involved if the 
firm agreed to participate. Additionally, each Associate 
representative with whom the researcher visited was given a 
brief written resume"^ of the proposed research, which included 
the estimated duration of each interview to be conducted and 
the total time commitment by the organization (Appendix D). 

Of the initial ten organizations contacted, all but 
one agreed to participate in the study. The one exception was 
an organization which had begun exploring MIS applications 
very recently, and felt, therefore, that there would not be 
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any projects for study in the organization which staisfied the 
researcher *s project selection specifications* However, to 
offset this organization's not participating in the study, one 
of the Associate representatives visited desired that two 
separate divisions of his organization be included in the 
study. The researcher agreed to this after satisfying himself 
that, for all practical purposes, the two highly autonomous 
divisions of this very large and diversified corporation were 
separate and independent with respect to the factors relevant 
to the study. 

Having gotten agreement to participate in the study 
from those organizations selected for the sample, a schedule 
was prepared for actual data collection. This schedule, cover- 
ing the period from early October to mid-December, 1970, allowed 
at least three days for collecting data in each organization. 
Each organization was again contacted by telephone to arrange 
the dates for interviewing in the organization. The only 
appointment made at this time was with the director of the 
information systems function or other individual(s) whom had 
been designated by the Associate representative to coordinate 
data collection in the organization. 

Data Collection Procedures 

The data collection procedures followed in each 
organization which participated in the study were as follows: 
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1. An Initial meeting with the manager of the 
information systems function and/or his 
designated representatives was opened by 
explaining precisely what data were to 
be collected, and from whom, after a 
general description of the research study 
had been provided. Next, the organization 
representatives were asked to suggest 
projects for study which met the project 
selection specifications described earlier 
in this chapter. This proved to be a more 
time consuming task than expected, and, in 
some cases, required as long as three hours 
rather than the thirty minutes estimated 
from the pretest experience. Those two 
projects which best satisfied the selection 
specifications were chosen for study. The 
organization representatives were then 
requested by the researcher to arrange 
appointments with the project leaders of 
the two projects selected. Finally, the 
manager of information systems, or his repre- 
sentative, was asked to complete pages 1-3 of 
the questionnaire. This was the only part of 
the questionnaire which was not completed during 
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interviews with the researcher. However, on 
several occasions the researcher did assist 
the respondents for pages 1-3 by clarifying 
points that were confusing, or in making certain 
that the respondents interpreted the questions 
as the researcher intended. 

2. The meeting with the project leader was opened 
with a brief description of the nature of the 
study and the data to be collected, and with 
assurances to the project leader that all of 
his responses would be held in confidence by 
the researcher. The remainder of the 1^ to 3 
hours spent with the project leader was devoted 
to completing the appropriate portions of the 
questionnaire. The respondent was given a 
copy of the questionnaire to read as the 
researcher asked the questions. 

Upon completion of his portion of the question- 
naire, the project leader was requested to 
arrange appointments with at least two managers 
in the area(s) which received the products of 
the project. For this purpose, a staff analyst 
or similar person was considered to be a manager. 
In general, if a user was involved in making 
planning and/or control decisions in a functional 
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area of responsibility, he was considered a 
manager whether he had any subordinates or not. 
Finally, the project leader was requested to 
arrange an appointment with the computer opera- 
tions manager, or other knowledgeable individual 
in the computer operations function who could 
answer those questions pertaining to computer 
operations for the project. 

3. The meeting with each user was opened in exactly 
the same manner as the meeting with the project 
leader. After the preliminary comments by the 
researcher about the study, the remainder of 

the fifteen to thirty minutes spent with the 
user was devoted to the completion of the 
relevant portions of the questionnaire. As 
with the project leader, the user was provided 
a blank questionnaire to read as the researcher 
asked the questions. 

4. The same preliminary explanations on the nature 
of the study, and so forth, were provided to 
the computer operations management. In most 
cases, the computer operations data for both 
projects in an organization were collected in 
one interview. This interview generally lasted 



from ten to fifteen minutes. 
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Steps 2 and 3 of the above procedure were repeated for the 
second project studied in the organization. 

Data Analysis Procedures 

After all the data had been collected in all ten 
organizations in the sample, the interview forms were reviewed 
and coded by the researcher. Since the questionnaire was 
originally developed to facilitate coding and conversion of 
the data to punched card form, this was a relatively easy task. 

Several methods of data analysis were considered, 
but it was decided that Kendall's (1962) tau statistic provided 
the best measure available of relationships among the variables 
for the following reasons:^ 

1. The tau statistic is robust and relatively 
powerful in dealing with ranked data contain- 
ing ties among ranks. 

2. Being a measure of relationship among ordinal 
variables, there are no assumptions concerning 
distribution of the raw data for the tau statistic. 

3. The S statistic, which is actually the numerator 
of the tau statistic, and is a measure of the 
agreement or disagreement between a pair of 
rankings, is distributed approximately normally 

^For discussions of the tau statistic, its distribution 
under various conditions, and corrections for continuity, see 
Burr, 1960; Goodman and Kruskal, 1954; Hoffding, 1947; Kendall, 
1947; 1962; Sillitto, 1947; Somers, 1962; and Whitfield, 1947. 
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for N ^ 9. The S statistic can, therefore, 
be converted to a normal deviate, Z(s ) , after 
computing the variance of S. This normal 
deviate can then be tested, using the normal 
probability table, to ascertain the signi- 
ficance level of the relationship between 
two rankings* 

4* since the data collected could be arranged 
in naturally ordered bivariate tables, the 
computation of tau, S, and Z(s) could be 
accomplished quite easily using the UMST 
(1969) programs available for the University 
of Minnesota's CDC 6600 computer system* 

The first step taken in analyzing the data collected 
was to prepare a descriptive profile of the sample* This 
descriptive profile was broken into two parts: descriptive 

statistics on the organizations in the sample, and descriptive 
statistics on the projects* These descriptive statistics 
consisted of means, medians, and ranges on those variables which 
were amenable to such treatment, and distributions of scores 
on the variables which Were not. These descriptive statistics 
on the organizations and the projects are presented in Chapter 5. 

The second step in analyzing the data was to compute 
the tau statistics for relationships among the variable rankings* 
This step, in turn, was divided into three substeps. 
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First, it was necessary to determine whether or not 
those variables which were originally intended to be composites 
of several separate items were, in fact, valid composites repre- 
senting the same factor, or were, in reality, separate factors 
which should not be aggregated. The variables in question 
were User Par ticipa tion-PL (Var. 41), User Participation-User 
(Var. 46), User Satisfaction-User (Var. 54), project leader's 
perception of success, and operations success. To make this 
determination, Pearson product-moment correlation coefficients 
were computed for all of those separate items comprising a 
supposed aggregate factor. This computation was accomplished 
with computer program UMST 530. If any item within a supposed 
composite factor was not correlated at least .50 with every 
other item in that composite factor, that item was considered 
to be a separate measure and dropped from the composite factor. 
It was found that Variables 41, 46, and 54 did, in fact, have 
items which were intercorrelated at least .50, and, therefore, 
were acceptable as composite measures of the factors in 
question. However, the intercorrelations of separate items 
in the assumed composite variables representing the project 
leader's perception of success and computer operations success 
were so low as to preclude their being considered composite 
factors. Consequently, computer operations success and the 
project leader's perception of success were treated in all 
further analysis as two separate variables (59 and 60), and 
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four separate variables (55-58), respectively. (See Appendix E 
for descriptions of all variables used and their scoring.) 

Next, in order to determine roughly which variable 
rankings were related enough to warrant further investigation 
of the relationships, UMST 540 was used to provide a tau 
statistic for every possible cross -classification among the 
61 variables in the study. The tau statistic computed by 
UMST 540 was based on Siegel's (1956) formula for tau-B, and 
gave an inflated normal deviate where there were ties present 
in the rankings. This inflated normal deviate resulted from 
not making any correction for continuity prior to computing 
the normal deviate. However, the computation with UMST 540 was 
relatively simple and fast, and provided an excellent means for 
identifying those variable relationships which should be explored 
further in the last substep. 

Finally, UMST 590 was used to derive tau, S, and 
Z(s) statistics for all criterion variables cross-classified 
against each other, all hypothesis variables cross-classified 
against all criterion variables, and all other relationships 
which were indicated as possibly significant from the previous 
run based on Siegel's formula for tau-B. UMST 590 computations 
are based on Kendall's (1962) formulas for tau-B, tau-C, S, 
the variance of S, and the normal deviate of S. (See Appendix 
F for these computing formulas.) 
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PART III 



RESULTS OF THE STUDY 

Part III contains the substance of the research 
study. It is divided into three chapters. Chapter V 
presents descriptive data on the organizations in the sample 
and on the projects studied. These descriptive data consist 
of means, medians, and ranges where appropriate, and of 
distributions of responses by category where they are 
appropriate. 

Chapters VI and VII present the results of the 
statistical analysis of the sample data. The primary means 
of analysis was, as discussed in Chapter IV, the determination 
of association among variables using Kendall's rank correlation 
statistic, tau. The tables of association are included with 
the text to facilitate reader reference. 
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CHAPTER V 

DESCRIPTION OF THE STUDY SAMPLE 

Organizations in the Sample 
The ten organizations which participated in the study 
were all located in the Twin Cities area. Two of the organi- 
zations were highly autonomous divisions of a large, multi- 
business corporation. The remaining eight organizations were 
completely unrelated. 

The ten organizations were categorized into the 



following industry groupings: 

1. Manufacturing « .5 

a. Consumer products using 
continuous process 

production technology • .3 

b. Industrial products using 

assembly line/mass production 
technology 2 

2 . Non-manuf ac tur Ing 5 

a. Wholesale, retail trade 1 

b . Transpor ta tion 1 

c. Finance. 1 

d. Utility I 

e. Conmodity merchandising 

and processing 1 



Measures of Organization Size 

The organization sample represented considerable range 
in terms of relative organization size. This fact is reflected 
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in Table 1, which shows the mean, median, and range for four 
measures of organization size. 



TABLE 1 

Measures of Organization Size 





Mean 


Median 


Range 


Assets (Var. 1) thousands 
(7 organizations only) 


$593,286 


$170,000 


$ 60,000- 
$2,875,000 


Sales (Var. 2) thousands 
(8 organizations only) 


$647,000 


$491,000 


$ 76,000- 

$2,600,000 


Employees (Var. 3) 


9,931 


6,100 


560- 

47,900 


Customers (Var. 4) 

(8 organizations only) 


147,693 


27,500 


500- 

450,000 



Throughout Part III, the variable number is presented whenever 
a variable is discussed or entered in a table. This has been 
done to facilitate reader reference to Appendix E where all of 
the variables in the study are fully described. No particular 
significance should be attached to the variable numbers them- 
selves, however. The numbers were assigned by the researcher 
in essentially the order the variables appeared in the study 
questionnaire (Appendix B). 



Other Relevant Organization Attributes 

Several other attributes of the organization sample 
besides organization size are relevant to the present study. 
Included in this category are the reported degree of organiza- 
tional change over the last three years; the reported degree of 
centralization of the organization's MIS activities; the 
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organizational location of the manager of the information 
systems function; the organization's hardware investment and 
costs; and the size of the systems and programming staff. The 
data for all of these attributes are shown in Tables 2-4. 



TABLE 2 

Organization Change^ Centralization, and 
Level of the Information Systems Manager 



Number of 

Organizational Attributes Organizations 

• Organization Change over Last Three Years (Var.5): 

Little or no change 5 

Minor changes 1 

Considerable change 4 

•Centralization of Organization MIS Activities 
(Var. 8): 

All design, analysis, and programming 
performed by corporate MIS staff 
regardless of origin of project. ... 8 

Exactly as above, except that small opera- 
tions research group developed some 
of its own projects 2 

• Level of Information Systems Manager (Var. 9): 



Number of hierarchical levels between 
manager directly responsible for 
information systems function and 
the top operating executive: 

0 Levels 3 

1 Level 5 

2 Levels 2 

• Immediate Superior of Manager of Information 
Systems: 

Top operating executive 3 

Administrative vice-president . • 3 

(or similar position) 

Controller 3 

Market research director 1 
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TABLE 3 

Hardware Investment and Costs 



Mean Median Range 

Hardware Investment/Sales (Var. 6) .94% .90% .107«-2.1% 

Hardware Expense/Budget (Var. 7) 35% 357. 137.-50% 



TABLE 4 

Size of Systems and Programming Staff 





Mean 


Median 


Range 


Number In Systems and Programming 


50 


35 


7-156 


(Var. 10) 








Systems Staff/Total Employees 


1.31% 


.527. 


. 257.-8.2% 


(Var. 61) 
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Projects in the Sample 

This section of Chapter V deals with the descriptive 
findings relative to the twenty projects themselves. The 
variables to be discussed Include those pertaining to project 
scope and size, project personnel, project procedural features, 
and project success. 

Before taking up the specific variables for which 
data were collected, several general points concerning the 
project sample are In order. First, It Is reiterated that only 
projects which met the criterion of being MIS projects were 
Included In the sample. Projects which were merely data 
processing applications with no management Information spinoffs 
were rejected. In some Instances the satisfaction of this 
criterion severely narrowed the range of projects available for 
study. In fact. In some organizations It was only after consider- 
able probing that valid MIS projects could be Identified. This 
situation led to accepting projects for study which did not fall 
within the desired time and size parameters given In Chapter IV. 
Specifically, the following deviations were made from the 
criteria for project selection: 

1. It was desired that all projects In the sample 
had had at least two people who worked primarily 
on the project during Its active development 
periods. Eighteen of the projects met this 
requirement, but two did not. Two projects 



were developed entirely by one person. 
However, they were of such scope that they 
were considered to be valid components of 
the sample. 

It was desired that all projects had been 
completed between six and twenty-four 
months prior to the time of the study. 

This proved to be the most difficult 
requirement to satisfy. In that only 
fourteen projects were completed between 
November, 1968, and July, 1970. Three 
projects were completed prior to November, 
1968, the earliest of which was implemented 
in September, 1967. This means that a little 
more than three years had elapsed between 
the time the project was completed and the 
time the data were collected. While the 
period of recollection was thus over a 
year longer than the maximum recollection 
period desired. It was impossible to 
estimate how much the data were biased by 
Increased forgetting of the specifics of 
the project. This same observation holds 
for all of the projects beyond the twenty- 
four month limit, and, Indeed, for all of 
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the projects in the sample, excepting the 
very recent ones. 

Three projects were completed after July, 

1970, the most recent of which was imple- 
mented in October, 1970. The primary consid- 
eration with such recent projects was, of 
course, whether or not the users had had 
adequate time to evaluate the results of 
the projects. The researcher was satisfied 
that two of the projects completed since July, 
1970, did have an adequate experience base 
for fair evaluation. One of these projects, 
completed in September, resulted in a daily 
output which had provided ample opportunity 
for shakedown and appraisal. The other 
project, completed in August, had been 
evolving over a period of twelve months, 
with very close interplay between the users 
and the systems staff. In effect, this 
project was not considered completed until 
the users had indicated satisfaction with 
the products after several months of evaluat- 
ing alternative outputs. The third, and most 
recent project, completed in October, 1970, 
was studied in early December. To check on 
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the users* perceptions of the project after 
two additional months experience, a follow-up 
was made with the users In early February, 

1971. Although still two months short of 
the desired six months minimum, there Is no 
reason to suspect bias enough to warrant not 
Including the project in the sample. 

General Types of Projects 

As previously stated, all of the projects studied 
were management Information system projects; that is, outputs 
were generated for a manager at some level which were Inputs 
to his decision-making process. This general classification 
between MIS projects and data processing projects was further 
broken down into the subcategory of best fit. The subcategories 
and numbers of projects in each are shown in Table 5. 

Origin of Projects and Nature of Oblectives 

Of the twenty projects studied, 657« were initiated 
by user managements. This would seem to indicate a desire on 
the part of functional managers to utilize the computer as an 
aid in carrying out their decision-making responsibilities. 
However, the reported objectives in initiating projects did 
not always seem to be as clear and specific as they might 
have been. These observations are based on the distributions 
of project origins and project objectives shown in Table 6. 



TABLE 5 



Types of MIS Projects Studied 



Number of 
Projects 

Types of MIS Projects of Type 

Models ; projects Involving management 

science techniques such as simulation, 

mathematical programming, or forecasting; 

generally projects aimed at providing 

inputs to planning processes 4 

2 . Data processing spinoffs ; projects 
which evolved from operational control 
systems at the lover levels of the 
organization; generally spinoffs from 
accounting and logistics systems, and 
aimed at providing input to control 
processes through monitoring, triggered, 
or demand reports (see Blumenthal, 1969, 

p. 51) 9 

3. Data collection and analysis ; projects 
which were developed from the ground up 
with specific uses and requirements; 
generally Involved setting up means to 
collect desired Information, creating 
the necessary data base, and developing 
analytical routines to manipulate the 
data base; these projects aimed at 
providing Inputs to both planning and 
control processes; an example of such a 
project would be a marketing Intelligence 
s ys tern 
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TABLE 6 

Project Origins and Objectives 



Number of 

Pro jects 

Origin of Project (Var. 11) 

User 13 

Top level management 3 

Information systems staff 4 

Project Objectives (Var. 12) 

Specific, measurable, written objectives. . . 5 

Specific, non-measurable, written objectives. 6 

General, clear, written objectives 2 

General, unwritten objectives 3 

Rather vague objectives 4 



Project Scope and Size 

The following variables relate to the size and scope 
of the projects in the sample: 

Complexity - (Var. 13) 

There was no objective measure of comparative 
complexity of projects in the sample. The 
ratings of project complexity were entirely 
relative to the individual project leaders and 
the systems environments in their organizations. 

For this reason, the distribution given below is 
not the distribution of complexity of all sample 
projects based on some objective scale applied 
to all projects; rather, it is a distribution of 
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the ratings of complexity assigned by each 
project leader relative to his own experience: 
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Highly complex 3 

Above average complexity . . .5 

Average complexity 8 

Below average complexity . • .2 

Very low complexity 2 

Size and composition of project staffs 

Table 7 provides a breakdown of the composition and 

size of project staffs for the projects in the sample: 

TABLE 7 

Composition of Project Staffs 









Mean 


Median 


Range 


Number 


on 


Project (Var. 17) 


5.0 


4 


1-12 


Number 


of 


Analysts (Var, 18) 


1.5 


1 


1-3 


Number 


of 


Programners (Var. 19) 


2.5 


2 


1-8 


Number 


of 


Users - N=13 (Var. 20) 


1.8 


2 


1-3 


Outside consultants used on three 


proiects 







Time spent on projects 

Table 8 contains data on the elapsed time and the man 
months spent on the projects in the sample: 
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TABLE 8 



Time Spent on 


Projects 








Mean 


Med Ian 


Range 


Elapsed Months (Var. 21) 


12.1 


10.5 


2-48 


Man Months (Var. 22) 


22.0 


10.0 


4-87 



Attributes of Project Personnel 

The following variables relate to the education, 
systems experience, and organization experience of those who 
worked directly on the projects in the sample: 

Systems experience 

Table 9 contains data on four measures of the 
systems experience of those who worked directly 
on the projects studied: 

TABLE 9 

Systems Experience of Project Personnel 



Mean Median Range 



Systems Experience of Project Leader •- 

months (Var. 24) 66.2 43.5 0-300 

Mean Systems Experience of Project 
Personnel Including Users-- 

months (Var. 25) 47.3 28.5 8-180 

Mean Systems Experience of Project 

Personnel from MIS Staff Only- 

months (Var. 26) 52.5 37.5 11-180 



Two or More Years Systems Experience-- 

proportion (Var. 27) 58% 



50% 0-100% 
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It should be noted that the values given in Table 9 
for Variables 25, 26, and 27 are not those for all 
project personnel in the sample. Rather, they 
are the values derived from the means or proportions 
that Variables 25, 26, and 27 represented for each 
project. Thus, 47.3 months is the mean of twenty 
project means, not the mean systems experience for 
all individuals who worked on projects in the sample. 
Impact of systems experience - (Var. 28) 

With respect to the impact of systems experience on 
project success, project leaders generally indicated 
that such experience was a contribution to project 
success, and lack of systems experience hindered 
success somewhat. However, as the following 
distribution of Variable 28 shows, 257. of the 
project leaders felt that prior systems experience 
was of no importance one way or the other to 



project success: 

Experience critical to success 4 

Experience contributed somewhat 

to success 8 

Experience of no importance 5 

Lack of experience of some staff 

members detrimental 3 

Education 



Table 10 contains data on three measures of the 
educational levels of those who worked directly 
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on the projects studied: 

TABLE 10 

Formal Education of Project Personnel 









Mean 


Median 


Range 


College Degree - 
proportion 




(Var. 30) 


65% 


75% 


0-100% 


Two Years College 
proportion 


- 


(Var. 31) 


79% 


87% 


33-100% 


Mean Years Formal 


Education 

(Var. 32) 


15.3 


15.3 


13-18 



As with systems experience, it should be noted that 
the above values are derived from project statistics, 
not individual measures. 

Organization experience 

Table 11 contains data on two measures of the 
experience project personnel had in their respect- 
ive organizations at the beginning of the projects 
s tudied, 

TABLE 11 

Organization Experience of Project Personnel 



Mean Median Range 



Mean Years Organization 



Experience 




(Var. 33) 6.75 


6.75 


1.5-20.0 


Two or More Years 
Experience - 


Organization 

proportion 


(Var. 34) 69% 


69% 


25-100% 
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As with systems experience and education, the above 
values are derived from project statistics, not 
Individual measures. 

Turnover - (Var. 29) 

Of the twenty projects In the sample, only three 
experienced any turnover among the project staff. Two of 
these projects had a turnover rate of 50%, while the third 
project's turnover rate was 677». For the three projects 
which experienced staff turnover, the project leader's 
appraisal of the Impact of this turnover on project success 



was distributed as follows: 

Very detrimental 1 

Somewhat detrimental 1 



Contributed to success ... .1 

Procedural and Organizational Attributes of Projects 

Tables 12 and 13 contain breakdowns of the study 
sample according to several organizational and procedural 
attributes of the projects. In addition to the Information In 
these tables, several amplifying or explanatory comments 
concerning the attributes shown are In order. 

As can be seen from Table 12, thirteen of the projects 
studied (65%) were developed by project teams comprised of user 
representatives working with Information systems staffs. Of 
those thirteen, two project teams had user representatives 
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As with systems experience and education, the above 
values are derived from project statistics, not 
individual measures. 

Turnover - (Var. 29) 

Of the twenty projects in the sample, only three 
experienced any turnover among the project staff. Two of 
these projects had a turnover rate of 507o, while the third 
project's turnover rate was 677o. For the three projects 
which experienced staff turnover, the project leader's 
appraisal of the Impact of this turnover on project success 



was distributed as follows: 

Very detrimental 1 

Somewhat detrimental 1 



Contributed to success ... .1 

Procedural and Organizational Attributes of Projects 

Tables 12 and 13 contain breakdowns of the study 
sample according to several organizational and procedural 
attributes of the projects. In addition to the information in 
these tables, several amplifying or explanatory comments 
concerning the attributes shown are in order. 

As can be seen from Table 12, thirteen of the projects 
studied (65%) were developed by project teams comprised of user 
representatives working with information systems staffs. Of 
those thirteen, two project teams had user representatives 
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assigned full time during the analysis and design phases; the 
remaining eleven project teams had user representatives working 
on the projects on a part-time basis. Further, for eleven of 
the thirteen projects where a team was utilized, the user 
representatives who participated on the teams were also respon- 
sible for implementing the completed projects in their respective 
departments, and for maintaining liaison with the information 
systems staffs after implementation. For the other two 
projects, team participants during development were not the 
ones responsible for implementation and continuing liaison* 

For the thirteen projects which were developed by 
teams, the mean value for the proportion of total project man- 
months accounted for by user personnel (Var. 15) was IS?*; the 
median value was 18.57*; and the range was 4-517*. 

Finally, project leaders generally felt that user 
representatives on project teams made valuable contributions 
to project success. Nine of the project leaders who headed 
teams appraised user member contributions as very great, as 
critical to project success. The other four project leaders 
felt that user members made some contributions, and that the 
projects would not have turned out as well as they did without 
the users. 

With respect to project documentation, formal standards 
for documentation were considered to exist even if such 
standards were applicable to only the programming phase of a 
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projecC. It should also be pointed out that the quality of 
project documentation was not measured against any universal 
objective standard or scale. Rather, the documentation quality 
distribution shown In Table 12 reflects the appraisals of 
Individual project leaders as to how good they thought the 
project documentation was for their respective projects. 

The degree of user participation In the analysis 
and design of each project was appraised by both user personnel 
and the project leader. The amount of agreement between those 
appraisals will be taken up In a subsequent chapter. However, 
It Is worth noting at this point that there was a tendency 
for fairly high levels of user participation In the projects 
studied, whether viewed from the users' side or the project 
leader's perspective. This observation Is supported by the 
statistics In Table 13. 



TABLE 12 

Project Procedural and Organizational Attributes 



Number of 
Projects 



Project Team (Var. 14): 

Yes 13 

No 7 

Combination Analys t/Progranmer (Var. 23): 

Combination 9 

Separate 11 

Documentation Standards (Var. 35): 

Yes 16 

No 4 
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TABLE 12 (Cont.) 

Project Procedural and Organizational Attributes 



Number of 
Prolec ts 



Quality of Documentation (Var, 36): 

High quality 6 

Adequate 8 

Somewhat Inadequate 5 

Low quality 1 

Project Control (Var. 42): 

Formal or Informal report of progress 

against plan required of project 

staff at least monthly 11 

No regular progress reporting 

required of project staff 9 

Programming Language (Var. 43): 

COBOL 15 

FORTRAN 2 

Assembly 3 

Generalized Software Problems (Var. 44): 

No problems 15 

Minor problems 4 

Major problems 1 



TABLE 13 

User Participation 



Mean Median Range 



User Participation -PL (Var. 41) 

possible range: 3-15 11.5 12.5 4-15 

User Participation-User (Var. 46) 

possible range: 40-200 153 162 83-200 
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Proiect Success 

It was stated In Chapter III that four criteria of 
project success were defined for the study: time, cost, user 

satisfaction, and computer operations problems. The performance 
of the projects on each of these criteria Is shown In Table 14. 

In addition to what Is shown In the table, the following comments 
concerning the criteria are relevant: 

Time 

Of the twenty projects, six were completed within 
the times estimated; the remaining fourteen 
exceeded their time estimates by varying amounts 
up to 900%. Of the six projects which were 
completed on time, overtime was required to do so 
for three of them. One of the three on>tlme 
projects which had considerable overtime also 
had substantial programmer resources added to 
the staff to meet the time deadline. 

Cost 

An interesting finding regarding project cost 
was that nine of the twenty projects In the 
sample had no cost budget. No cost estimates 
were ever nude, or at least ever recorded or 
referred to later, for nearly half of the 
projects studied. Since It was Impossible to 
determine If a project was completed within Its 
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cost budget If such a budget never existed, 
the cost data for these projects were estimated, 
based on man«inonths data and estimates of 
computer test time. This seemed to be a 
reasonable approximation to project cost, 
since by far the largest part of project cost 
was accounted for by manpower cost, which. 

In turn, was a direct function of man-months 
spent on a project. Where program test time 
was determined, through discussions with the 
project leader, to be of any consequence In 
the total project cost, a factor for computer 
cost was Included In the cost estimates. 

Using, then, the best estimates that could be 
devised for project cost data. It was found 
that only two of twenty projects were completed 
within their cost budgets. The remaining 
eighteen projects exceeded their cost estimates 
by varying amounts up to 500%. 

User satisfaction 

Since five separate factors were aggregated 
together to form the measure of user satisfac- 
tion (Var. 54), the range of possible values 
for this measure was 50-250, with 50 represent- 
ing extremely low user satisfaction and 250 
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representing extremely high user satisfaction. 

As the statistics in Table 14 indicate, a 
fairly high level of user satisfaction existed 
for most projects in the sample. If the sum 
of the middle of the five possible ratings on 
the scale for each of the five factors was 
taken as an "average" level of user satisfaction 
the resulting "average" value would be 150. 

It can be seen that both measures of central 
tendency shown in Table 14 for the actual 
sample well exceeded that hypothetical value. 
Computer operations problems 
In general, the projects in the sample were 
successful in terms of not creating problems 
in the computer operations function. With a 
five point scale, ranging from very serious 
problems at the low end to no problems at the 
high end, a hypothetical average would have 
fallen in the middle of the scale, or at a 
value of 3, which represents moderate problems 
for computer operations. However, the mean 
value for computer operations problems (Var. 59) 
in the sample was 3.95, and there was no 
project in the sample rated below 3 on the scale 
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The distribution of the ratings on this 
criterion was as follows: 

No problems 5 

Very minor problems. . . 9 
Moderate problems. ... 6 

TABLE 14 
Project Success 







Mean 


Median 


Range 


Actual Time /Estimated Time 


(Var. 48) 


209.6% 


139.57. 


75-9007. 


Actual Cost/Estimated Cost 


(Var. 51) 


194.77. 


151.57. 


82-5 007. 


User Satisfaction-User 

(possible range: 50-250) 


(Var. 54) 


175.8 


192.5 


80-220 


Computer Operations Problems 
(possible range: 1-5) 


(Var. 59) 


3.95 


4.0 


3-5 



Summary 

In summary, it can be seen from a review of the 
descriptive information presented in this chapter that the 
study sample represented considerable diversity among organiza- 
tions and projects. The organizations in the sample were from 
varied industries; they represented a wide range in terms of 
size; and they differed in varying degrees on several other 
organizational attributes relevant to the study. With respect 
to the individual projects studied, there were three general 
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types of MIS projects: models; projects that were spinoffs 

from data processing applications; and projects which were 
developed from the ground up to provide data collection and 
analysis capability in specified planning and control areas. 
The twenty projects represented substantial diversity in 
terms of size and scope; procedural and organizational 
attributes; and, finally, performance on the criteria of 
success • 
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CHAPTER VI 
TESTS OF HYPOTHESES 

Introduction 

This chapter and the one which follows are devoted 
to presenting the results of the analysis of the sample data* 

As indicated in Chapter IV, the primary form of statistical 
analysis was the computation of Kendall's rank correlation 
statistic, tau. This measure of the degree of relationship 
between the rankings of two variables was tested for statistical 
significance in all cases* 

The number of possible cross-classifications, and 
resultant relationship statistics, with 61 variables was 3660. 

To avoid burdening the reader with all of these statistics, 
and to conserve computation time and cost, the steps discussed 
in Chapter IV were followed* The results of the data analysis 
have been further limited here by presenting only the following: 

1. Relationships among criterion variables; 
all tau statistics presented regardless 
of significance levels* 

2. Relationships of hypothesis variables 

to each of the four criteria of success; 
all tau statistics presented regardless 
of significance levels. 

3. Relationships of criterion variables to 
other non-hypothesis variables which were 
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slgnlf leant at the ,10 level or beyond. 

4, Relationships of hypothesis variables 
to other non-criterion variables which 
were significant at the .10 level or 
beyond. 

3* Other relationships among variables 
felt by the researcher to be relevant 
to the study, which contributed to 
understanding of the overall conclusions 
drawn, and which were significant at the 
.10 level or beyond. This last category 
is presented in Chapter VII. 

All tests for the significance of the relationship 
between the rankings of two variables were made with the .10 
level as the cutoff. Any relationship with a probability of 
chance occurrence greater than .10 was not considered to be 
significant. Since the direction of expected relationship 
was stated in only those cases where a hypothesis was tested 
(category 2 above), all significance tests were two-tailed 
tests except those involving the relationship of a hypothesis 
variable to a criterion variable. 

Relationships Among Criterion Variables 

At the time this study was conceived, it was assumed 
that the four criteria to be used were Independent. Otherwise, 
all four would not have been included in the study. However, 
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no hypothesis to this effect was set up. For this reason, 
all tests of relationship among the four criteria are two- 
tailed tests. 

It can be seen from Table 15 that there were no 
relationships among any of the criteria which reached signifi- 
cance at the .10 level. This supports the assumption of 
independence among the four criteria used in the study. 

TABLE 15 

Relationships among Criterion Variables^ 



User Satis- 

Cost (Var. 51) faction (Var. 54) 



Operational 
Problems (Var. 59) 



Tau 



Zisl 



?(zy 



Tau 



ZisjL 






Tau 



z(«) 



im: 



Time 

(Var. 48) 
Cost 

(Var. 51) 

User Sat- 
isfaction 
(Var. 54) 



.257 



1.534 



.124 



.064 



.091 



.359 



.522 



.720 



.604 



.067 



-.030 



.345 



.281 



.105 



1.582 



.778 



.916 



.104 



All values shown for tau in all tables are tau-C unless a given 
value is prefixed by B, in which case that value is tau-B. 
See Appendix F for differences in computation of tau-B and 
tau-C, 



2 

P(Z) value shown includes both tails of the normal distribution. 
For all tables, the value shown under P(Z) represents the 
chance probability, one-tailed or two-tailed, as indicated, 
of having a normal deviate for the S statistic as large as 
the one shown under Z(s). 
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The strongest relationship between any two criterion 
variables was the one between user satisfaction-user (Var. 54) 
and operations problems (Var* 59), with a tau of ,345. This 
relationship had a chance probability of *104, which almost 
reached the *10 cutoff level established* Although not signifi- 
cant based on the arbitrary *10 cutoff used in the study, there 
did appear to be a meaningful relationship between how satisfied 
the user was and how successfully the project had been implemented 
by computer operations* Where there were substantial problems 
for computer operations in running particular programs, or in 
meeting certain schedules, there also tended to be lower user 
satisfaction* It cannot be assumed that computer operations 
difficulties caused user dissatisfaction, although this did 
appear to be the case with two of the projects studied* Rather, 
the conclusion drawn was that those projects which created 
problems for the computer operations area were also somewhat 
unsatisfactory in the eyes of the users of the outputs. In 
other words, there was a tendency for shortcomings in the 
systems development work for these projects to be reflected 
in both inadequate planning for impact on computer operations 
and poor response to user information needs* 

Tests of Hypotheses 

Tables 16-19 provide a complete picture of the relation- 
ship between each hypothesis variable and each criterion variable. 
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TABLE 16 

Success in Terms of Time 
Actual Ttme/Es clmated Time (Var. 48) 



Rank 


Var. 


Tau 


Z(s) _ 




1 


User Participation -User 


46 


-.181 


1.061 


.145 


2 


Measurable Project Objectives 


12 


-.219 


1.142 


.127 


3 


Project Team 


14 


-.130 


.477 


.316 


7 


Level of Information Systems 
Manager 


9 


-.292* 


1.358 


.087 


8 


Two or More Years Systems 
Experience 


27 


.203 


1.193 


.116 


12 


Documentation Standards 


35 


-.380* 


1.753 


.040 


13 


Project Control 


42 


-.153 


.763 


.223 


14 


Turnover 


29 


.082 


.529 


.300 


16 


Originator 


11 


-.037 


.155 


.439 


17 


Centralization 


8 


Not tested by this 


sample 


20 


Two or More Years Organization 
Experience 


34 


.297* 


1.740 


.041 


23 


High Level Programming Language 


43 


-.315* 


1.774 


.039 


31 


Mean Years Formal Education 


32 


.630* 


3.471 


.0003 


32 


Combination Analyst/Programner 


23 


.420* 


1.562 


.060 


33 


Systems Staff /Total Employees 


61 


.291* 


1.659 


.050 


34 


Hardware Investment/Sales 


6 


-.063 


.330 


.378 



* - Significant at the .10 level or beyond - one-tailed 
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TABLE 17 

Success In Terms of Cose 
Actual CosC/EsClmaced Cost (Var. 31) 



Rank 


Var. 


Tau 


^s)_ 


P(Z) 


1 


User Participation -User 


46 


.154 


.894 


.185 


2 


Measurable Project Objectives 


12 


.100 


.503 


.309 


3 


Project Team 


14 


.210 


.794 


.214 


7 


Level of Information Systems 
Manager 


9 


-.240 


1.107 


.135 


8 


Two or More Years Systems 
Experience 


27 


-.104 


.596 


.275 


12 


Documentation Standards 


35 


-.200 


.899 


.185 


13 


Project Control 


42 


-.220 


1.109 


.134 


14 


Turnover 


29 


.030 


.158 


.437 


16 


Originator 


11 


.187 


.930 


.177 


17 


Centralization 


8 


Not tested 


by this 


sample 


20 


Two or More Years Organization 
Experience 


34 


.055 


.295 


.384 


23 


High Level Programming Language 43 


-.097 


.519 


.302 


31 


Mean Years Formal Education 


32 


.108 


.567 


.287 


32 


Combination Analyst/Progranmer 


23 


-.010 


.000 


.500 


33 


Systems Staff/Total Employees 


61 


.263* 


1.491 


.069 


34 


Hardware Investment/Sales 


6 


-.023 


.099 


.461 



* - Significant at the .10 level or beyond - one-tailed. 
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TABLE 18 

Success In Terms of User Satisfaction 
User Satisfaction-User (Var, 54) 



Rank 


Var. 


Tau 


Z.(s)___. 


P(Z) 


1 


User Participation-User 


46 


.440* 


2.618 


.005 


2 


Measurable Project Objectives 


12 


>.081 


.403 


.345 


3 


Project Team 


14 


.080 


.278 


.390 


7 


Level of Information Systems 
Manager 


9 


.105 


.464 


.322 


8 


Two or More Years Systems 
Experience 


27 


-.126 


.729 


.234 


12 


Documentation Standards 


35 


.090 


.379 


.352 


13 


Project Control 


42 


-.207 


1.040 


.149 


14 


Turnover 


29 


-.217* 


1.480 


.069 


16 


Originator 


11 


.420* 


2.132 


.017 


17 


Centralization 


8 


Not tested 


by this 


sample 


20 


Two or More Years Organization 
Experience 


34 


.253* 


1.477 


.069 


23 


High Level Programning Language 43 


.067 


.346 


.366 


31 


Mean Years Formal Education 


32 


.222 


1.201 


.115 


32 


Combination Analyst/ Programmer 


23 


.420* 


1.561 


.060 


33 


Systems Staff/Total Employees 


61 


.229* 


1.293 


.098 


34 


Hardware Investment/Sales 


6 


.057 


.296 


.383 



* - 



Significant at the *10 level or beyond - one-tailed 
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TABLE 19 

Success in Terms of Computer Operations 
Computer Operations Problems (Var, 59) 



Rank Var. 


Tau 




. nzi 


1 


User Participation-User 


46 


.180 


.820 


.206 


2 


Measurable Project Objectives 


12 


-.007 


.000 


.500 


3 


Project Team 


14 


-.050 


.170 


.433 


7 


Level of Information Systems 
Manager 


9 


B-,285* 


1.340 


.090 


8 


Two or More Years Systems 
Experience 


27 


-.255 


1.176 


.120 


12 


Documentation Standards 


35 


-.170 


.811 


.210 


13 


Project Control 


42 


-.060 


.261 


.397 


14 


Turnover 


29 


B .000 


.000 


.500 


16 


Originator 


11 


B-.147 


.665 


.254 


17 


Centralization 


8 


Not tested 


by this 


sample 


20 


Two or More Years Organization 
Experience 


34 


.157 


.707 


.241 


23 


High Level Programming Language 


43 


B-.157 


.695 


.243 


31 


Mean Years Formal Education 


32 


.330* 


1,542 


.062 


32 


Combination Analys t/Programmer 


23 


.060 


.204 


.420 


33 


Systems Staff/Total Employees 


61 


.037 


.143 


.443 


34 


Hardware Investment/Sales 


6 


.232 


1.063 


.144 



* - Significant at the .10 level or beyond - one-tailed. 
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Each of these tables shows the relationships between all sixteen 
of the variables representing hypotheses to be tested and one 
criterion variable* The sixteen hypothesis variables are shown 
in the same order as in the ranking in Chapter IV; the number 
to the left of each variable is the rank of that variable from 
the original list (Figure 2, Chapter IV). 

Rather than merely repeat in narrative form what is 
contained in Tables 16-19, the comments here will be restricted 
to general observations about the findings shown in the tables, 
and discussions of those hypotheses which were significantly 
related to at least one criterion: 

1. Each of the hypotheses was represented by one 
variable in the statistical analysis. Appendix 
E should be consulted for a description of what 
each variable is a measure of, and how it is 
scored. Of the sixteen hypotheses, all but 
one were tested by the data from the sample. 

Variable 8, representing the degree of centrali- 
zation of the information systems function, was 
dropped from the analysis because of the lack of 
spread on the variable among the organizations in 
the sample. Eight organizations were scored at 
the highest level of centralization, and the 
remaining two were at the next to highest level, 
a result solely of small operations research 
groups which did some progranmlng work. 
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2. Of the fifteen hypotheses which were tested, 
five were found not significantly related to 
any criterion* Those five were: 



2 - Measurable project objectives 


- Var. 12 


3 - Project team 


Var. 14 


8 - Two or more years systems 
experience 


Var. 27 


13 - Project control 


Var. 42 


34 - Hardware investment/sales 


Var. 6 



The fact that those hypotheses which were not 
significantly related to any criterion were from 
the top, middle, and bottom of the ranked listing 
is of of some interest. At least for the sample 
involved in this study, those factors believed 
by the SMIS conference respondents to be most 
crucial to MIS project success did not prove to 
be so critical in the case of two of the three 
top ranked factors. Also of considerable interest 
was the lack of significant relationship between 
reported project control efforts and any criterion 
of success used in the study. This last finding 
will be explored further at a later point. 

3. Of the ten hypothesized relationships which were 
found to be significant at the .10 level or 
beyond, four were cases where a hypothesis variable 
was significantly related to only one criterion 
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varlable. The remaining six hypothesis 
variables were related to two or three 
criterion variables* 

4* Looking at the tables from a criterion 
perspective, the time success criterion 
was significantly related to more hypothesis 
variables (7) than was any other criterion 
variable. User satisfaction was signifi- 
cantly related to six hypothesis variables, 
computer operations success to two, while 
cost success was significantly related to 
only one hypothesis variable. 

Hypotheses Significantly Related to Project Success 

Each of the hypothesis variables which was significantly 
related to at least one criterion of project success is dealt 
with briefly below: 

1 - User Participation-User (Var. 46). As hypothesized, 
the higher the level of perceived user participation 
in design and development of a project, the greater 
the success of that project as viewed by the user. 

The statistical relationship between these two 
variables was very strong in the sample. However, 
perceived user participation in project develop- 
ment was not significantly related to any of the 



other three criteria of success. 
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7 - Level of Informatton Systems Manager (Var. 9). 

The higher the level of managers of the infor- 
mation systems departments in the organizations 
sampled, the less successful projects were in 
terms of time, and the more successful they were 
in terms of computer operations. Although there 
were no data from the study which explained these 
relationships very satisfactorily, it could be 
that the higher the level of the top computer 
executive in the organization, the less the 
felt pressure in the information systems function 
to meet time budgets. With respect to computer 
operations problems being less where the top 
computer executive was at a higher level in the 
organization, one possible explanation would be 
the apparent greater attention to computer 
operations requirements when the information 
systems function is accorded higher status in 
the organization. Put differently, where the 
organization has placed emphasis on using the 
computer effectively, as evidenced by locating 
the top computer executive close to top manage- 
ment, there may be more emphasis on assuring 
that computer operations requirements are 
considered in systems design. 




*1 




f » 






* 



u 





9 



i 









r 






I 








« k 



• .. 





-102 



12 - Documentation Standards (Var. 35). Where 

formal documentation standards were reported 
to have been applicable to projects in the 
study, the time success tended to be greater 
than where there were no such standards. It 
was impossible to determine whether the documen- 
tation standards themselves actually contributed 
to better performance in terms of time, or the 
existence of such standards merely reflected 
an atmosphere of greater control over informa- 
tion systems activities. In any case, this 
finding should be Interpreted with caution due 
to the great extent of ties in the ranking of 
the dichotomous documentation standards variable. 

14 - Turnover (Var. 29). As was pointed out in 
Chapter V, only three of the projects in the 
sample experienced any turnover of project 
personnel. Therefore, the negative relationship 
of turnover to user satisfaction must be inter- 
preted very cautiously due to the great extent of 
ties in the turnover ranking. A close review 
of all the variables for those three projects 
with turnover led to the conclusion that the 
negative relationship with user satisfaction 
was a chance one, and should not be given much 
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credence. The reason for this conclusion 
was that in the case of only one project 
did low user satisfaction follow at all 
from project staff turnover. In that one 
case, the turnover was among key user repre« 
sentatives on the project team. 

16 • Originator (Var. 11). The hypothesis that 
success is greater for projects originated 
by users, as opposed to those initiated by 
top management or the information systems 
staff, was confirmed where success was viewed 
in terms of user satisfaction. This would 
appear to stem in part from a greater feeling 
of participation among the users who originated 
projects, and, in part, from a greater clarity 
in what the users wanted when they initiated 
projects themselves. Where the users knew 
what they wanted, and participated in develop- 
ing it, their satisfaction with the results 
tended to be greater. 

20 - Two or More Years Organiaation Experience (Var. 34). 
The organization experience of the project staff 
was significantly related to two criteria: time 

and user satisfaction. In the case of time, the 
greater the organization experience of the project 
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staff, the poorer the time performance. 

This would seem to reflect a greater tendency 
for those who knew their organizations well 
to spend more time delving into the various 
ramifications and possibilities with projects 
once development was underway. This would 
likely draw out the time spent on the projects 
beyond what was originally anticipated. In 
the case of user satisfaction, the positive 
relationship to organization experience of 
the project staff indicated that the greater 
the proportion of those working on a project 
who understood the organization they were serving, 
the better the results in the eyes of the users. 

It is worth noting here that organization 
experience was not related at all to the 
measure of user participation. It might 
have been assumed that high organization 
experience would have reflected high propor- 
tions of user representatives on project teams, 
which, in turn, would have contributed to 
high levels of perceived user participation. 

Since user participation was highly related 
to user satisfaction, the assumption would 
then be that this last relationship was 
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real ly what was being picked up in a 
relationship between user satisfaction 
and organization experience of the 
project staff. This was not the case, 
however. 

23 - High Level Programming Language (Var. 43). 

From the data in the sample, the hypothesis 
of greater project success with higher level 
programming languages was confirmed with 
respect to the time criterion. Where COBOL 
was the programning language used, the 
projects were completed in less time 
relative to estimates than where FORTRAN or 
assembly language was used. However, it 
should be noted that the types of projects 
were different where FORTRAN was used, and 
this in itself influenced the time performance. 
FORTRAN was used with modeling applications 
which were evolutionary in nature. Through 
trial and error processes, the models were 
developed to the point the users were satis- 
fied with the results. This situation 
resulted in very extended development periods 
for these projects. 

31 - Mean Years Formal Education (Var. 32). The 



mean education level of the project staff was 
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very strongly related to the time criterion 
of success* What this indicated was that 
the higher the education of those who 
worked on projects, the poorer the perform- 
ance in terms of time* The only apparent 
explanation for this relationship was a 
tendency for those with more formal 
education to delve more deeply into 
possible enhancements and embellishments 
on the projects they worked on* Again, the 
influence of modeling applications should 
be considered in this connection* Those 
engaged in modeling applications generally 
had high levels of formal education* It 
will be recalled that the modeling projects, 
because of their evolutionary nature, were 
drawn out and usually way over the original 
estimates for time to completion. 

Also of significance was the relationship of 
project staff education levels to computer 
operations success* One explanation for 
this relationship appeared to be the tendency 
for those with the greatest amounts of formal 
education to make provisions for the efficient 
computer implementation of their projects* 
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32 - Combination Analys t/Progranmer (Var, 23). 

Where combination analys t/programmers were 
used to develop projects, the time performance 
was relatively poor, but user satisfaction was 
relatively high. While no explanation was 
readily apparent for the poor time performance 
where combination analys t/programmers were 
used, the high user satisfaction seemed to 
be attributable to the ability of the user 
to look to one person for any problem that 
arose. Where the user had to deal with an 
analyst in some cases, and with a programmer 
in others, there appeared to be user frustra- 
tion and less satisfaction with the project 
as a whole. 

33 - Systems Staff/Total Employees (Var. 61). 

Although they were all rather weak relationships, 
the relative size of the organization's systems 
staff was related to three separate criteria. 
Positive relationships to both the time and 
the cost criteria would seem to indicate that 
there was more slack in those organizations 
with large systems staffs. Where large staffs 
existed, the pressure to meet time and cost 
budgets or estimates did not seem to be as great. 
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As a consequence, time and cost performance 
were poor. However, user satisfaction was 
higher for those projects developed in organi- 
zations with larger systems staffs. Taken 
all together, the pattern would seem to be one 
of substantial organizational commitments to 
effective use of the computer as an aid to 
managerial decision-making, manifested through 
relatively large systems staffs which were 
oriented toward giving the users what they 
wanted at the expense of time and cost overruns. 

Further Comments Concerning the Criteria of Success 
Data were collected for several other variables 
besides the criteria and the hypotheses to be tested. To provide 
a more complete picture, all significant relationships between 
these other variables and the criterion variables are shown in 
Tables 20-23, These tables, and certain relevant non-quantitative 
findings, are discussed in this section. 

Time Success (Var. 48). Table 20 contains those 
relationships between time success and other variables in the 
study which were significant at or beyond the .10 level. As 
might be expected, the projects with the greatest elapsed time 
had the poorest time success. Also, the higher the proportion 
of college degrees on project staffs, the poorer the time perform- 
ance. This is merely another way of looking at the educational 
level, which was discussed earlier. 






» I » ^ 




»M •• • 






II 










I 



- 109 - 



TABLE 20 

Actual Time/Estimated Time (Var, 48) 
(Two-tailed significance at or beyond ,10 only) 



College Degree 
Elapsed Months 
User Management Interest 
Time Success -PL 



Var. 


Tau 


Zls± 


PCZl 


30 


.550 


3.281 


.0014 


21 


.515 


3.081 


.002 


39 


-.367 


1.864 


.062 


49 


-.437 


2.328 


.020 



From the data in the study, it appeared that high 
interest and involvement of top level user management contributed 
to better time performance. Where the top management levels of 
user organizations were actively engaged in promoting projects, 
those projects seemed to get completed faster relative to time 
estimates. This finding has interesting implications which 
tie in to some comments to be made later concerning the reward 
structures for systems people in organizations. 

Another finding shown in Table 20 was the accuracy 
of project leader's perceptions concerning the success of their 
projects in terms of time. It should be borne in mind, however, 
that this relationship is based on asking the project leader 
only how successful he felt the project was in terms of time, 
with no consideration given to his views of other aspects of 
project success. This point, too, will be dealt with later 
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at greater length. 

Cost Success (Var. 51). Table 21, consisting of 
only one entry, provides the same sort of confirmation of 
the project leaders* perceptions about project cost success 
as described just above for time success. Again, this point 
will be dealt with later in the context of global project 
success, and what that seemed to be comprised of in the 
eyes of project leaders. 



TABLE 21 

Actual Cos t/ Estimated Cost (Var. 51) 

(Two-tailed significance at or beyond ,10 only) 

Var. Tau Z(s) P(Z) 

Cost Success-PL 52 -.644 3.444 .0006 



In addition to the questions asked of project leaders 
which were tied to some kind of response scale, there were two- 
open-ended questions which permitted project leaders to state 
in their own words the reasons they felt their projects came 
out poorly on time and/or cost performance. These comments 
were reviewed by the researcher and classified according to 
eight response categories. The categories, and the number of 
projects falling into each, were as follows: (there were 

multiple responses for most projects) 



-Ill- 



1. Poor initial estimates; especially 

inadequate was the time allowed for 
implementation 10 

2. Inexperience of the project staff 

with this particular type of applica- 
tion or language 7 

3. Key people on the project were doing 
several things at once; competing 

demands for their time caused delays 4 



4. The project was allowed to evolve over 



a long period of user learning and 

growth; no attempt was made to freeze 

requirements at one point and then 

consider the job done when those 

requirements were programmed • • 4 

5. File handling problems; delays caused 
by waiting for data base development 

or accessibility 3 

6. Poor computer test turn-around time 2 

7. Turnover of project staff . 2 

8. Project was too large and complex 

to be managed o2 



From the above, it would appear that the greatest 
problem in meeting time and cost budgets, at least, in the 
eyes of project leaders, was poor estimates of time and cost 
in the first place. Actually, most of the other categories 
are related in some way to the first one. In estimating time 
to complete a project, the experience of the project staff 
should have been considered, as should the expected availability 
of computer test time, and so forth. This is not to say that 
every contingency could have been foreseen and provided for 
in making initial estimates, but it would seem that some factors 
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which did cause delays were not given adequate consideration 
at the time estimates were made. 

One final point concerning project leaders* comments 
about time and cost performance is worth mention. In two 
cases ^ the project leader had no direct authority over programmers. 
This created communications problems which resulted in considerable 
confusion as to what was to be programmed, with attendant delays. 
Both of the projects were in the lower half of every criterion 
ranking. 

User Satisfaction (Var. 54). A close scrutiny of the 
users* ratings of their satisfaction with the results of the 
projects in the study revealed several interesting points. It 
will be recalled that the measure of user satisfaction was a 
composite variable, made up of five separate items which were 
highly intercorrelated. However, one of the components of 
the user satisfaction variable, a measure of implementation 
problems, was also treated as a separate variable (53) for 
comparative purposes. As Table 22 shows, the users viewed 
the ease or difficulty of implementing a completed project 
as a very important aspect of the perception of project success.^ 

^While there is bias in the value of tau shown in Table 
22, due to the inclusion of the implementation problems 
variable in the user satisfaction variable, it is apparent 
that implementation was a large factor in the minds of users 
in evaluating the success of projects. 
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TABLE 22 

User Satisfaction-User (Var. 54) 

(Two-tailed significance at or beyond .10 only) 





Var, 


Tau 


Z(3) 


im. 


Implementation Problems- 


User 


53 


.720 


4.192 


.0004 


User Satisfaction-PL 


57 


.656 


3.584 


.0004 


Project Success-PL 


56 


.607 


3.124 


.002 


Computer Operations 


Cos t 


60 


.347 


1.796 


.074 


Satisfaction of Objectives- 


PL 


55 


.320 


1,687 


.092 


Man Months 


22 


-.280 


1.668 


.096 



It is also noteworthy that the variable representing 
the project leader's perception of implementation problems (58) 
was not significantly related to user satisfaction. This last 
observation will be pursued further at a later point, but it 
is apparent that users viewed implementation problems differently 
than project leaders viewed them. This conclusion is supported 
by the very strong relationships between user satisfaction and 
three variables representing the project leader's evaluation 
of project success. These three variables, shown in Table 22, 
were the project leader's perception of how satisfied the 
users were with the project results (57), how successful the 
prajiect leader felt the project was overall (56), and how well 
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the project leader thought the original objectives for the 
project had been satisfied (55). In other words, users and 
project leaders agreed very well on the relative success of 
a project, but implementation problems were a factor in the 
users' evaluations while they were not a factor in the project 
leader's evaluation of project success. 

Another very interesting aspect of user satisfaction 
came to light in looking at the relationships among the 
various criteria. It was felt that users might have been 
dissatisfied with projects which took too long to complete; 
that they disliked waiting for something they felt they needed. 
To get at this question, the individual projects that had 
the worst performance on time were checked for user satisfaction 
scores. It was already known that these two criteria were 
not significantly related for the whole study, but it was felt 
that looking at the extremes might reveal something that was 
hidden in the sample statistics. The surprising result of 
this investigation was that the project ranked last on time 
success was ranked third on user satisfaction, and the project 
ranked next to last on time success was ranked second on user 
satisfaction. In other words, two of the top three projects 
in terms of user satisfaction were the two bottom projects 
on time success. Upon closer review, what appeared to be 
underlying this finding was the evolutionary nature of the 
two projects in question. Both projects involved rather 
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complex models which were developed over a fairly long period 
of trial and error. Rather than taking a set of user stated 
desires or needs, and developing some programs to satisfy them, 
the systems staffs concerned worked very closely with the users 
in evolving those products which best fit the users* needs. 
There seemed to have been tacit recognition that the users 
would learn as they went, that they would become more sophis- 
ticated as they worked with outputs at successive stages of 
development, and would, therefore, not be satisfied with a one- 
shot development effort. Although the users were highly 
satisfied with what they ultimately received from the projects, 
this evolutionary approach took much longer than had originally 
been anticipated. 

Based on the above findings, an effort was made to 
discover other evidence in the data which supported what has 
been called the evolutionary approach to project development. 

It was hypothesized that projects which had not been developed 
with such an evolutionary strategy would reflect the following 
characteristics : 

Users who were originally satisfied with project 
results at the time the projects were completed 
might now be less satisfied with what they were 
receiving. This shift in satisfaction over time 
would have resulted from a user learning process 
which was not matched by any enhancements in the 
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information system outputs they were receiving. 

In other words, as the users became more sophis- 
ticated through working with the products of the 
projects in question, their information needs 
would shift. If the information system products 
did not shift with this increasing sophistication, 
the users would now be less satisfied with the 
projects than when they were first implemented. 

To test the above hypothesis, two separate components 
of the user satisfaction variable were analyzed. These 
components represented a measure of how well the user felt the 
original project objectives had been met, and a measure of how 
satisfied the user now was with the products of the project. 

It was already known that these two components of overall user 
satisfaction were highly correlated. However, it was not known 
if these two factors were scored at about the same level on a 
project by project basis. To get at this latter question, the 
mean value and the standard deviation were computed for each 
factor. These statistics were then used in testing the differ- 
ence between the two means to determine its significance. The 
mean value for the factor measuring how well original objectives 
were reportedly met was 39.2 with a standard deviation of 9.8. 
The mean and standard deviation for the reported current satis- 
faction factor were 33.5 and 10.3, respectively. The difference 
between these two means was 5.7, which, when tested by the t 
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statistic with 38 degrees of freedom, was significant beyond 
the .10 level for a two-tailed test. 

Although the t test described above was retrospective, 
and, therefore, not generalizable beyond the sample, it was 
interpreted as an indication that the users in the study did shift 
their perceived information requirements fairly quickly after MIS 
project implementation. While users may have scored a project 
very high on how well it served their information needs when 
first implemented, they tended to score present satisfaction at 
a lower level. This interpretation would seem to be a good 
argument for the evolutionary approach to MIS projects since 
such projects have been defined as management-decision oriented. 

Finally, with respect to user satisfaction. Appendix G 
contains comments by users relative to what they believed should 
have been done to improve the projects in the study, and what 
they view present shortcomings to be. These comments are not 
direct quotes; rather, they represent the notes taken by the 
researcher as the users responded to an open-ended question in 
the questionnaire (Appendix B, page 222). 

Computer Operations Problems (Var, 59). As reflected 
in Table 23, the measure of computer operations success used in 
the study was related to a secondary measure of such success, 
namely, the operations cost of a project. This relationship 
indicates that projects which were implemented most smoothly 
by computer operations were also the most successful in terms 
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statistlc with 38 degrees of freedom, was significant beyond 
the .10 level for a two-tailed test. 

Although the t test described above was retrospective, 
and, therefore, not generalizable beyond the sample, it was 
interpreted as an indication that the users in the study did shift 
their perceived information requirements fairly quickly after MIS 
project implementation. While users may have scored a project 
very high on how well it served their information needs when 
first implemented, they tended to score present satisfaction at 
a lower level. This interpretation would seem to be a good 
argument for the evolutionary approach to MIS projects since 
such projects have been defined as management-decision oriented. 

Finally, with respect to user satisfaction. Appendix G 
contains comments by users relative to what they believed should 
have been done to improve the projects in the study, and what 
they view present shortcomings to be. These comments are not 
direct quotes; rather, they represent the notes taken by the 
researcher as the users responded to an open-ended question in 
the questionnaire (Appendix B, page 222 ). 

Computer Operations Problems (Var. 59). As reflected 
in Table 23, the measure of computer operations success used in 
the study was related to a secondary measure of such success. 



namely, the operations cost of a project. This relationship 
indicates that projects which were implemented most smoothly 
by computer operations were also the most successful in terms 
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of computer operating costs. This relationship is not 
surprising, and would appear to be the underlying 
explanation of the relationship between user satisfaction 
and operations cost success shown in Table 22. It will be 
recalled that the relationship of user satisfaction to 
computer operations success was discussed earlier. 

TABLE 23 

Computer Operations Problems (Var. 59) 

(Two-tailed significance at or beyond ,10 only) 

Var. Tau Z(s) P(Z) 
Computer Operations Cost 60 .420 2.078 ,038 



Further Comments Concerning the Hypothesis Variables 

Tables 24 through 37 contain relationships of the 
hypothesis variables to other non-criterion variables for 
which data were collected in the study. Each relationship 
tabled will not be dealt with in detail. Rather, patterns 
that were detected, and findings of particular interest, will 
be dealt with. The discussions which follow are oriented 
around the hypothesis variables, and are presented in the 
same sequence in which these variables appeared in the 
earlier chapter dealing with the testing of the hypotheses. 

User Participation-User (Var, 46). User participation 



in project development, as perceived by user personnel, was 
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related to more separate variables than any other variable 
in the study, as shown in Table 24. However, several of these 
relationships tended to cluster together, revealing what 
appeared to be more basic underlying relationships. 

TABLE 24 

User Participation-User (Var. 46) 

(Two-tailed significance at or beyond .10 only) 



User Satisfaction-PL 
Project Success -PL 
Satisfaction of Objectives -PL 
Time Success -PL 
Assets 

Specificity of User 
Requirements -PL 

Measurable Project Objectives 

Complexity 

User Participation-PL 

Specificity of User 
Requirements -User 

Hardware Inves tment/Sales 

Implementation Problems -User 

Two or More Years Systems 
Experience 



Var> Tau Z(s) P(Z) 



57 


.644 


3.530 


.0004 


56 


.553 


2.887 


.004 


55 


.513 


2.767 


.006 


49 


.506 


2.737 


.006 


1 


.417 


1.904 


.058 


38 


.400 


2.160 


.030 


12 


.394 


2.112 


.036 


13 


.369 


2.009 


.046 


41 


.367 


2.174 


.030 


45 


.356 


2.111 


.034 


6 


.309 


1.772 


.076 


53 


.291 


1.701 


,090 


27 


B-.353 


1.984 


.048 
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User participation was significantly related to 
four separate measures of how successful the project leader 
perceived the project to be (Variables 49, 55, 56, and 57) « 

The underlying relationships here would appear to be the 
relationship of user participation to user satisfaction, and 
the relationship of user satisfaction to the project leader's 
perceptions of success. One interesting exception was the 
relationship of user participation to the project leader's 
perception of time success. It appeared that where perceived 
user participation was high, project leaders tended to discount 
the actual time performance in evaluating time success. This, 
in turn, implies that project leaders were more oriented to 
how the users viewed the projects than to the actual time 
spent compared to time estimated. This observation will be 
dealt with in more detail further on. 

Those users in organizations which were large in 
terms of assets, and which had high computer hardware to sales 
ratios, felt that they participated more in project development 
efforts. This probably reflected a stage of development in 
computer usage which the larger, more heavily computer oriented 
organizations had reached. 

With respect to the reported specificity of project 
requirements, and the reported clarity of project objectives, 
user participation was perceived to be higher when the measures 
of these variables were higher. It was difficult to determine 
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precisely what was behind these relationships, but it seems 
that users felt they participated to a greater extent where 
projects were well defined and objectives were clearly stated 
from the beginning. The specificity and clarity of objectives 
would Indicate that users knew what they wanted and were in a 
position to actively work with the systems staffs in getting 
it. Where objectives were vague, and the users weren't really 
certain what they were after, the Information systems staffs 
picked up the ball and did more or less as they saw fit. 

This is not to impugn the information systems staffs* Rather, 
given somewhat vague requirements, they had to fill in the 
gaps themselves to get something running, relegating the users 
to ques t ion -answerers when the systems people got stuck. The 
important Implication in these conclusions would seem to be the 
necessity for clear objectives and specific information require- 
ments from users if the users want to be heavily involved in 
the development effort, thus providing greater assurance that 
the project results will satisfy their information needs. 

Perceived user participation was also related to 
having fewer implementation problems. This logically would 
follow from closer user Involvement during project development, 
thereby providing the users with a more precise understanding 
of what was to be implemented and possible attendant problems. 



Finally, one relationship shown in Table 24 which 
may appear somewhat odd was the one between perceived user 
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participatlon and the systems experience of the project staff. 
Being a negative relationship, one might conclude that staff 
personnel with greater systems experience did not permit much 
user participation. What actually seems to have been the case, 
however, was that user members of project teams, having little, 
if any, systems experience, pulled the systems experience 
measure down. Since having user personnel on project teams 
generally contributed to higher perceptions of user involve- 
ment, the negative relationship between participation and 
systems experience resulted. 

Measurable Project Objectives (Var. 12). There were 
two main clusters of variables related to the reported clarity 
of project objectives. The first cluster, consisting of 
Variables 1, 2, 3, and 6 in Table 25, represent organization 
and computer installation size. The larger organizations, and 
those most heavily committed to computerized information process- 
ing, tended to have the most specific and measurable project 
objectives. This, again, would seem to indicate a more advanced 
stage of computer usage. 

The second cluster of variables (14, 27, 39, 41, and 
46) seems to represent the effects of clear project objectives 
on other aspects of project development. Where project object- 
ives were reported to have been clearly stated from the beginning 
of a project, the users appeared to be more involved in all 
subsequent project efforts, a point made earlier. Project teams 
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were more common, top Level user management took a more active 
Interest In what was hapi>enlng, and user participation, as 
perceived by both the project leaders and users, was higher. 

All of these factors combined to present a picture of users 
knowing what they wanted and actively participating with the 
systems staff in getting it. In addition, where project object 
ives were reported to have been clearly stated in the beginning 
there were fewer course changes during project development, as 
indicated by the negative relationship between objectives and 
the measure of project change requests honored by the project 
staff (Var. AO). 

TABLE 25 

Measurable Project Objectives (Var. 12) 

(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


. Z(s) 


P(Z) 


Project Team 


14 


.580 


2.320 


.020 


Assets (14 projects) 


1 


.574 


2.507 


.012 


Employees 


3 


.437 


2.321 


.020 


Number on Project 


17 


.400 


2.138 


.032 


User Participation-User 


46 


.394 


2.112 


.036 


Sales (18 projects) 


2 


.386 


1.921 


.054 


User Management Interest 


39 


.347 


1.806 


.072 


User Participation-PL 


41 


.344 


1.830 


.068 


Hardware Investment/Sales 


6 


.312 


1.660 


.096 


Two or More Years Systems 
Experience 


27 


-.400 


2.146 


.032 


Changed as Requested 


40 


-.405 


1.922 


.054 
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An interesting exception to the above general 
conclusions was the case where project objectives were scored 
high, but the specific user requirements were scored low. There 
were three projects in this category. For these three projects, 
the original objectives were reported to have been quantified 
and clear. However, while the users apparently knew what they 
wanted to achieve, they didn't appear able to define the 
information content or format they needed to achieve it. In 
essence, project development seems to have begun before the 
users had analyzed their own information requirements well 
enough to define them clearly to the systems staff. The result 
in all three cases was considerable frustration among all 
concerned, and very poor time performance on the projects 
Involved. The point of all this is that specific user require- 
ments did not automatically follow from well specified object- 
ives, and where they did not, considerable difficulties were 
encountered in project development. The fact that users were 
able to state clearly what they wanted to achieve did not mean 
that they understood their own information-decision environments 
well enough to plot a course to get there. 

Project Team (Var. 14). It was expected that the use 
of a project team, composed of user representatives and informa- 
tion systems staff, would enhance the level of user participation 
in project development. This expectation was confirmed when 
user participation was seen through the eyes of the project leader. 
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Where there were teams > consisting in part of user personnel, 
project leaders felt that objectives were clear, user require- 
ments were more specifically stated, and user participation 
was quite high (see Table 26). 

TABLE 26 

Project Team (Var. 14) 

(Two-tailed significance at or beyond .10 only) 



Specificity of User 
Requirements 

User Participation-PL 

Measurable Project Objectives 

Number on Project 

Two or More Years Systems 
Experience 



Var. 


Tau 


Z(s) 


_ liZ) 


38 


.750 


3.032 


,0027 


41 


.660 


2.603 


.009 


12 


.580 


2.320 


,020 


17 


.500 


1.964 


.050 


27 


-.760 


3.018 


.004 



However, the measure of user participation representing the 
users* perceptions was not significantly related to the existence 
of a project team. This surprising finding for the whole sample 
led to a detailed review of each project in an effort to explain 
the divergence of the users* and the project leader *s views of 
ostensibly the same factor, user participation. This detailed 
review revealed the following: 

1. The three highest ranked projects on user 
participation (users* view) were all 
developed by project teams. 
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2* The two lowest ranked projects on user 
participation (users* view) were not 
developed by project teams, 

3, Four projects where teams were used, but 
which had low ranks on user participation 
(users* view), seemed to reflect consider- 
able disagreement among individual user 
respondents on how much they, as users, 
participated in project development. 

To explore item 3 above further, eighteen of the questionnaires 
were analyzed in terms of Individual user responses to the 
four questions that were aggregated in arriving at the user 
participation scores for these projects. Only eighteen projects 
were analyzed, since for two projects there was only one user, 
and no disagreement on the separate questions was possible. For 
each of the eighteen projects analyzed, an "index of disagreement" 
was computed as follows: 

For each of the four questions comprising the 
composite variable, user participation, the 
difference between two separate user responses 
was determined by merely subtracting the low 
response from the high response on the fifty 
point scale. These four differences were 
then summed together to get an index of 
disagreement for the project. For example. 
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i£ one user responded to the question 
"What was the degree of your organization's 
active participation throughout the evolu- 
tion of this project?" with a scale value 
of forty, and the other user responded 
with a scale value of twenty, the difference 
for that one question was twenty. That 
difference was then added to the remaining 
three differences computed in the same 
manner to arrive at the index value for the 
project. 

The mean value of the index of disagreement for all 
eighteen projects was 45.6. When the index of disagreement 
value for each of the four projects mentioned in item 3 above 
was compared to this mean index value, it was found that these 
four projects had the highest index of disagreement scores in 
the sample, with two at 70, one at 100, and one at 120. Further, 
all four of these projects had one thing in common: each was a 

case where one of the user personnel interviewed had actually 
worked on the project team and the other user had not. As 
would be expected, the individuals who had actually been on 
project teams scored user participation consistently higher 
than did the users who had not been on the project teams. 

What the above findings would seem to indicate is 



that users disagreed among themselves concerning how much they 
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participated in a project. While the individuals who were 
actually on the team felt that participation was high, other 
users often felt they had very little to do with the project. 
Where the objective of a project was to provide management 
information to several people in a user area, in fact, only 
those who actually were on the project team ended up getting 
what they wanted. Thus, while having a project team did seem 
to facilitate users feeling they participated to a larger 
degree, a team did not assure such feelings of participation. 
This situation was particularly obvious where the user represen- 
tatives on a team were staff personnel from the user area, but 
the actual recipients of the results of the project were other 
individuals who had not been very much involved in project 
development. 

With respect to the perceptions of project leaders 
concerning user participation, no distinctions were apparently 
made among the kinds of users who were on project teams. The 
mere presence of user representatives led to perceptions of 
high user participation, thus the high relationship between 
Variables 14 and 41 in Table 26. 

Another finding of interest relative to the use of 
project teams, was that user satisfaction with results was 
lower where the user members of project teams were not the 
ones responsible for implementing the completed projects, 
nor for maintaining on-going liaison with the information 
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systems staffs* There were two projects which fell in this 
category. Whereas the mean value of user satisfaction (Var. 54) 
for all projects in the sample was 175, the mean value of user 
satisfaction for the two projects cited was 122.5. Both of these 
projects were in the bottom half of the user satisfaction rank- 
ing, with ranks of 12 and 15. 

Finally, the fact that user representatives were 
full-time or part-time on project teams seems to have made no 
difference on any criterion. Having part-time or full-time 
user members on a project team appeared to depend more on the 
nature of the project and the structure of the organization 
than anything else. 

As would be expected. Table 26 shows that there 
tended to be more people involved in projects where teams were 
used. Also, the level of systems experience was lower where 
teams were used, since users, who had little or no systems 
experience, pulled the measure of systems experience down. 

Level of Information Systems Manager (Var. 9). For 
the seven organizations in the sample which provided asset 
data, the larger organizations, in terms of assets, tended to 
place the top information systems executive closer to the top 
operating executive of the organization. This is reflected in 
Table 27 as a negative relationship, since a lower value for 
Variable 9 meant fewer hierarchical levels between the informa- 
tion systems executive and the top operating executive. 



TABLE 27 



Level of Information Systems Manager (Var. 9) 
(Two-tailed significance at or beyond .10 only) 

Var. Tau S* P(s) 

Assets (N=7) 1 -.796 -13.0 .07 

* Since asset data were available for only 7 organizations, the 
normal deviate was replaced by the actual computed value of S. 
The probability level was determined from Slllltto (1947). 



Two or More Years Systems Experience (Var. 27). All 
of the relationships shown In Table 28 would seem to reflect, 
essentially, the degree of user Involvement In the projects In 
the study. There are no relationships there which are not 
consistent with the discussions already presented for other 
variables. Clear objectives, specific user requirements, 
user participation, the number of people on the project, and 
the use of a project team have all been shown to be related 
to each other. Since what was posited as underlying all of 
these relationships was user involvement. It Is not surprising 
that All of the variables shown In Table 28 were negatively 
related to the systems experience measure. Inasmuch as that 
measure was lower the more users were directly Involved In 



project development 
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TABLE 28 



Two or More Years 


Systems Experience* 


(Var. 27) 




(Two-talled significance at 


or beyond 


.10 only) 






Var. 


Tau 


ZlBl 


P(Z) 


Project Team 


14 


-.760 


3.018 


.003 


Specificity of User 
Requirements -PL 


38 


-.450 


2.453 


.014 


Measurable Project Objectives 12 


-.400 


2.146 


.032 


Number on Project 


17 


-.394 


2.311 


.022 


User Participation -PL 


41 


-.361 


2.141 


.032 


User Participation -User 


46 


B-.353 


1.984 


.048 


User SatlsfacClon-PL 


57 


-.319 


1.730 


.084 



*Slnce all values for tau are negative In this Cable, the 
entries are from roost negative (strongest relationship) to 
least negative (weakest relationship). 



Documentation Standards (Var, 35), Although the 
reported existence of formal documentation standards Is shown 
to be related to three variables In Table 29, great caution 
must be exercised In making any generalizations about these 
relationships. Only four projects In the study had no documen- 
tation standards, and two of these projects were In one 
organization. This organization was the smallest In the sample 
In terms of sales, and next to the smallest In number of 
employees. Further, all of the project staff members for both 
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of these projects held college degrees. A third project for 
which no formal documentation standards applied was In the 
smallest organization In the sample In terms of employees, and 
this organization was next to the smallest In sales. This 
third project, a sophisticated modeling application, was also 
developed by a team composed entirely of college graduates. 
These factors, together with the nature of the tau statistic 
computation, explain the relationships found. Therefore, 
generalizing In this case would be very hazardous. 

TABLE 29 

Documentation Standards (Var. 35) 

(Two-talled significance at or beyond .10 only) 





Var. 


Tau 


ZisJ__ 


P(Z) 


Sales (18 projects) 


2 


.543 


2.559 


.010 


Employees 


3 


.520 


2.149 


.032 


College Degree 


30 


-.490 


2.304 


.022 



Project Control (Var. 42). The primary conclusion one 
draws when looking at the relationships between the project 
control measure and other variables In the study Is that 
reported project control efforts were, essentially. Ineffective, 
perhaps even dysfunctional. It has already been shown In Tables 
16-19 that reported project control efforts were not related to 
any criterion of success. This would seem to Indicate that 
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project status Information, which was ostensibly collected for 
control purposes, was not. In fact, used to exercise control, 
at least over the project Itself. It Is worth noting that of 
the twenty projects In the study. In only one case did 
project status reporting result In additional resources being 
conmltted to the project. That one project was completed on 
time. In other projects, however. It appeared that project 
status Information was collected as "nice to know", or, at 
best, was used to keep others outside the Information systems 
function posted on when they would be affected by conversion 
or Implementation activities. This Is not to say that such 
overall organizational awareness Is not desirable nor necessary. 
But awareness and positive steps to Influence project develop- 
ment rate are not synonymous. 

Analysis of Table 30 reveals some further Interesting 
aspects of project control. Although reported project control 
was not related to time success. It was quite strongly related 
to the project leader's perception of time success. What this 
relationship seems to reflect was the tendency of the project 
leader to feel he had done his best where he was constantly 
maintaining close tabs on project status. He didn't feel the 
project had really been so bad In terms of time success , and 
tended to blame overruns on poor estimates, although In only 
one case did a project leader feel the original time estimate 
for the project had been unreallstlcl In fact, two projects, 
which exceeded time estimates by 600% and 900%, were rated 
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average In terms of time success, and three projects, which 
exceeded time estimates by 135%, 146% and 180%, were all rated 
above average on time success. 

TABLE 30 

Project Control (Var. 42) 

(Two-tailed significance at or beyond .10 only) 







Var. 


Tau 


Z(s) 


PXZ) 


Time Success -PL 




49 


.540 


2.861 


.004 


Originator 




11 


-.322 


1.722 


.084 


Quality of Documentation 


36 


B-.336 


1.650 


.100 


Changed as Requested 




40 


-.345 


1.683 


.092 


It appeared 


that project 


leaders 


felt an 


implicit 



pressure from tight project reporting requirements, but they 
felt helpless in doing anything about improving poor perform- 
ance beyond cutting corners in such areas as documentation and 
preparation for implementation. The negative relationship 
between reported project control efforts and the perceived 
quality of documentation would seem to lend support to this 
notion, as would the negative relationship between reported 
project control and perceived implementation problems. This 
latter relationship is not tabled, as it was not significant 
at the .10 level, but it was fairly strong, with a tau of -.307 
and a normal deviate of 1.638. 
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Another indication of the pressure felt by the 
project leader from project control schemes was the reported 
reluctance to make changes as development progressed. This 
was not necessarily a bad thing, but the pressure to get done 
as soon as possible seems to have prevented changes needed to 
provide users with more acceptable outputs in some cases. 
Although the reluctance to make desired changes did not seem 
to contribute to being on time with projects, it probably did 
prevent several of the projects from being more over estimate 
than they already were. 

Finally, as shown in Table 30, those projects 
initiated by the infottnation systems staffs appeared to be 
subject to more stringent control requirements than those 
initiated by users. It would seem that the planning process 
for project development was more explicit when a project came 
from the infottnation systems staff, resulting in more specific 
provisions for control of the project during its development. 
This observation opens up some very interesting possibilities 
with respect to kinds of information systems projects. These 
possibilities will be dealt with in Chapter VIII. 

Turnover (Var. 29). It was pointed out earlier that 
the turnover of project personnel could not be fairly evaluated 
because only three projects in the study experienced any turn- 
over at all. The same caution must be observed in considering 
the relationship of turnover to implementation problems, as 









l«ll 



1 ^^ 









§ 






« 













I 



• pi 








•• MC. • 















I 







I 








li#^ 






*#T. 












136 



viewed by the project leader » shown In Table 31, While the 
turnover that did occur on projects In the saoq>le appeared 
to create real difficulties for the project leaders of those 
projects, It would be risky to generalize beyond those projects 
The discussion of just what implementation problems seemed to 
consist of in the eyes of project leaders will be dealt with in 
the next chapter. 



TABLE 31 

Turnover (Var, 29) 

(Two-tailed significance at or beyond .10 only) 

Var. Tau Z(s) P(Z) 
Implementation Problems-PL 58 -.322 2.327 .020 



Originator (Var. 11). The relationship of reported 
project control efforts to origin of a project has already 
been discussed. The relationship of project origin to implemen 
tation problems, as viewed by the user, shown in Table 32, 
indicates that users felt they had less difficulty implementing 
projects they initiated than those initiated by higher level 
management or information systems staffs. This finding is not 
at all surprising, and fits the patterns of relationships 
already discussed. 

One finding that was surprising, however, was the 
absence of a significant relationship between origin of a 



project and the uacrs* perception of participation in project 
development. Although these two variables were related, with 
a tau of .29 and a normal deviate of 1.493, this relationship 
failed to reach significance at the .10 level. 

TABLE 32 

Originator (Var. 11) 

(Two'tailed significance at or beyond .10 only) 





Var. 


Tau 


ZCs) 


PCZi 


Implementation Problems 'User 


53 


.330 


1.709 


.088 


Project Control 


42 


-.322 


1.722 


.084 



A closer investigation of the sample data revealed 
what would appear to explain this situation. Two of the thirteen 
projects initiated by users had very low values for perceived 
user participation, 120 in both cases. This value was well 
below the overall mean for user participation, 152.6, and was 
far below the mean user participation value of 164 for those 
thirteen projects which were originated by users. The two 
projects in question, although originated by users, were 
developed, for the most part, without further substantive user 
participation. In one case, this appeared to be the result of 
the users not wanting to be bothered; and in the other case, the 
project leader simply did not work with the users, choosing to 
proceed on his own. It is worth noting that these two projects 
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were both very low in the ranking of user satisfaction, with 
one project ranked 17, and the other project ranked 18, 

Based on the above findings, it appears that the 
fact a user originated a project did not guarantee his partici- 
pation in its development. However, the results were not very 
satisfying to the user where participation did not follow 
from project initiation. 

Centralization of MIS Activities (Var. 8), This 
variable was not adequately tested in the present study. 

Two or More Years Organization Experience (Var, 34), 
There were few findings with respect to organization experience 
of the project staff which were of any significance. As might 
be expected. Table 33 shows that larger organizations, in 
terms of numbers of employees, had project staffs with less 
organization experience. This might be interpreted as an 
indication that more fluid personnel situations existed in 
larger organizations. Table 33 also shows that the education 
level of the project staff was related to organization experience. 
This finding would seem to run counter to the popular notion 
that people involved in systems work, and who are highly 
educated, have high job turnover rates. At least for this 
sanq>le, those who were more highly educated tended to have 
been in their current organizations for at least two years. 

Caution must be exercised in making too much of the above 
finding concerning turnover and education, however. The 
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measures used were not direct individual measures of Job 
longevity and education level. Rather, they were mean values 
and proportions, which were felt to be the best types of 
measures for the primary tasks in this study, but which were 
not the best measures for an analysis of turnover as it 
relates to education. 



TABLE 33 

Two or More Years Organization Experience (Var. 34) 
(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


Z(s) 


P(Z) 


College Degree 


30 


B.375 


2.132 


.034 


Mean Years Formal Education 


32 


.372 


2.046 


.040 


Employees 


3 


-.411 


2.401 


.016 



High Level Programming Langtiage (Var. 43). The 
relationship between the use of a higher level programming 
language and the quality of project documentation, shown in 
Table 34, reflects primarily the views of project leaders 
that the use of COBOL resulted in good program documentation. 
Since the perceived quality of program documentation appeared 
to be a key element in evaluating overall project documentation, 
this finding is not surprising. 
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TABLE 34 

High Level Progranmlng Language (Var. 43) 
(Two-tailed significance at or beyond .10 only) 

Var. Tau Z(s) P(Z) 
Quality of Documentation 36 .300 1.780 .076 



Mean Years Formal Education (Var. 32). The relationship 
found between project personnel education levels and organization 
experience has been dealt with to some extent in the discussion 
of the organization experience variable. As shown in Table 35, 
there were four other variables significantly related to project 
staff education besides organization experience. Looking at all 
five of these variables together, one finds a pattern that seems 
to explain their relationships to education levels of project 
staffs. The smaller organizations in the sample, in terms of 
numbers of employees, tended to have smaller project staffs whose 
members had high education levels, and who had been employed in 
their respective organizations for over two years. Further, 
smaller project staffs tended to have combination analyst/ 
programmers. Finally, users generally perceived implementation 
problems to be the least where combination analyst/programners 
developed projects (see Table 36). The above pattern was partic- 
ularly pronounced where applications dealing with mathematical 
models were involved. This does not mean that smaller organizations 



- 141 - 



had the most successful projects or the projects with the least 
implementation problems. Rather, the interrelations of the 
several variables discussed provide a possible explanation 
for findings relative to the education level of project personnel. 

TABLE 35 

Mean Years Formal Education (Var. 32) 

(Two-tailed significance at or beyond .10 only) 



Var. Tau Z(s) P(Z) 



Comb ina t ion Ana lys t/ Programmer 


23 


.440 


1.668 


.096 


Two or More Years Organization 










Experience 


34 


.372 


2.046 


.040 


Implementation Problems-User 


53 


.336 


1.884 


.060 


Number on Project 


17 


-.330 


1.821 


.068 


Employees 


3 


-.378 


2.073 


.038 



TABLE 36 

Combination Analyst/Programmer (Var. 23) 
(Two-tailed significance at or beyond .10 only) 



College Degree 
Implementation Problems-User 
Mean Years Formal Education 
Man Months 
Nijmber on Project 



Var. 


Tau 


Z(s) 


PCZl 


30 


.590 


2.238 


.026 


53 


.530 


2.029 


.042 


32 


.440 


1.668 


.096 


22 


-.500 


1.868 


.062 


17 


-.610 


2.306 


.022 
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Comblnatlon Analyst/Programner (Var. 23). The 
finding in the study that project staffs comprised of 
combination analyst/programmers had higher levels of education 
has already been mentioned In the preceding discussion of the 
primary measure of education level. Table 36 also shows that 
there were higher proportions of college degrees on project 
staffs composed of combination analyst/programners. This is 
merely one more way of looking at the same basic relationship. 

The tendency for users to feel Implementation 
problems were less with combination analyst/programmers has 
also been cited on several occasions. As was pointed out 
before, users seemed to feel frustrated and somewhat dissatisfied 
with Implementation efforts where they had to deal with differ- 
ent people from the Information systems staff, depending on the 
nature of the problems that came up. The ability to deal with 
one person who understood all aspects of the project, and who 
could respond to problems with action, was often mentioned by 
users as a very strong factor In their satisfaction with project 
results. It appears that combination analyst/programmers were 
better able to meet users' desires In this respect than were 
separate analysts or programmers. 

Finally, combination analyst/programmers generally 
meant fewer people on a project, and the projects they worked 
on took fewer man months. This latter relationship is open 
to Interpretation. It could be that since there were fewer 
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people working on projects where combination analyst/prograimers 
were used, the man months being less just reflected that under- 
lying relationship. On the other hand. It could be that there 
was less lost time and motion where combination analyst/ 
programmers were used, thus requiring fewer man months to 
complete a project. While this latter argument Is appealing 
from a logical standpoint. It Is hard to support In view of 
the fact that, as shown earlier In the discussion of crlterla- 
hypotheses relationships, project staffs composed of combination 
analyst/prograniners performed rather poorly on the time success 
criterion. Another possible explanation might be that combina- 
tion analyst/programmers were used on smaller, less complex 
projects. This position Is also difficult to defend, however, 
since there was little relationship between the measure of 
complexity (Var. 13) and the use of combination analyst/programmers . 
Perhaps the question could have been resolved had there been a 
comparable, objective measure of project complexity which was 
applied equally to all projects by the researcher. In the 
absence of such a measure, no satisfactory explanation could 
be found by the researcher for the relationship between using 
combination analyst/programmers and man months spent on a 
project. The whole question of combination versus separate 
analyst/programmers would seem to be a very fruitful one for 



further research. 
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Systems Staff/Total Employees (Var. 61). The 
measure of relative size of sample organizations' systems 
staffs was not significantly related to any other Independent 
variable with which it was cross -tabula ted. 

Hardware Investment/Sales (Var. 6). Organizations 
in the sample with the highest asset valuations also tended 
to have the highest ratios of computing hardware investment 
to sales for the past fiscal year. This relationship was 
computed for only the seven organizations which provided 
asset data, however (see Table 37). 

TABLE 37 

Hardware Investment/Sales (Var. 6) 

(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


Z(s) 


P(Z) 


Assets (N:7) 


1 


.784 


S*16.0* 


.02 


Measurable Project Objectives 


12 


.312 


1.660 


.096 


User Participation -User 


46 


.309 


1.772 


.076 



* Since asset data were available for only 7 organizations, 
the normal deviate was replaced by the actual computed value 
of S. The probability level was determined from Kendall (1962). 



The hardware investment/sales ratio was also related 
to the reported clarity of project objectives, and to user 
perceptions of participation in project development. It has 
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already been shown that perceived user participation was related 
to reported clarity of project objectives (Table 24). As was 
pointed out In the discussion of relationships shown In Table 
24, a high hardware/sales ratio was construed to represent a 
strong comnltment by an organization to computerized processing 
of Information for decision making. In such an environment, 

It appears that clear objectives were required before a project 
was Initiated, and users were more Involved In the planning for 
their own Information systems. 
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CHAPTER VII 

OTHER RELATIONSHIPS OF INTEREST 

Although the author's primary Intent in this thesis 
has been to report the results o£ conducting tests of the 
hypotheses posited for the study, the nature of the data 
collected has made it possible to investigate and report 
certain other relationships of relevance and Interest. Some 
of these "other" relationships have already been discussed 
in the last two sections of Chapter VI. This chapter focuses 
mainly on those findings which provide Insight into the ways 
project leaders viewed their respective projects. 

Project Leaders* Views on Project Success 

Tables 38-41 provide a picture of the way project 
leaders viewed project success in terms of time, cost, user 
satisfaction, and for the project overall. No attempt has been 
made to discuss every relationship shown in all four tables. 
Rather, what appeared to be meaningful patterns of relationships 
have been dealt with. 

First, although project leaders' evaluations of 
time and cost success were related to the actual measures of 
time success and cost success respectively, the most prominent 
factor underlying the overall evaluations of success appeared 
to be how satisfied the project leaders perceived the users 
were with the project results. To support this contention. 
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consider the results in Tables 40 and 41, When asked to rate their 
projects on how successful they were overall, the project leaders' 
responses were related to a cluster of other variables which repre- 
sented, essentially, user satisfaction. The only exception was 
the relationship between overall project success and time success. 

TABLE 38 



Time Success -PL 
(Two-tailed significance at 

Var. 


(Var. 49) 
or beyond 
Tau 


,10 only) 
Us)_ 


P(Z) 


Project Control 


42 


.540 


2.861 


.004 


User Participation-User 


46 


.506 


2.737 


,006 


Project Success -PL 


56 


.487 


2.578 


.010 


Cost Success -PL 


52 


B.447 


2.335 


.020 


Specificity of User 
Requlr emen ts -Us er 


45 


.394 


2.114 


.034 


User Satisfaction-PL 


57 


B.387 


2.006 


.044 


Elapsed Months 


21 


-.369 


1.963 


.050 


Actual Time/Estimated Time 


48 


-.437 


2.328 


.020 







TABLE 39 












Cost Success -PL 


(Var. 52) 








(Two-tailed 


significance at 


or beyond 


,10 only) 








Var. 


Tau 


Z(si ^ 


?iz±_ 


Time Success -PL 


49 


B.447 


2.335 


.020 


Number 


on Project 


17 


-.319 


1.707 


.088 


Actual 


Cost/Estimated Cost 51 


-.644 


3.444 


.0006 
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TABLE 40 

Project Success -PL (Var. 56) 
(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


Zis) 


pj;z) 


User Satis faction-PL 


57 


.713 


3.837 


.0002 


User Satisfaction-User 


54 


.607 


3.124 


.002 


Implementation Problems -User 


53 


.553 


2.920 


.004 


User Participation-User 


46 


.553 


2.887 


.004 


Time Success -PL 


49 


.487 


2.578 


.010 


Satisfaction of Objectives-PL 


55 


B.407 


1.979 


.048 


Specificity of User 
Requirements -PL 


38 


.367 


1.937 


.052 



TABLE 41 

User Satis faction-PL (Var. 47) 
(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 




P(Z1 


Project Success-PL 


56 


.713 


3.837 


.0002 


User Satisfaction-User 


54 


.656 


3.548 


.0004 


User Participation-User 


46 


.644 


3.530 


.0004 


Implementation Problems-User 


53 


.519 


2.871 


.004 


Satisfaction of Objectives-PL 


55 


.447 


2.469 


.014 


Specificity of User 
Requirements -PL 


38 


B.409 


2.116 


.036 


User Participation-PL 


41 


.394 


2.135 


.032 


Time Success-PL 


49 


B.387 


2.006 


.044 


Two or More Years Systems 
Experience 


27 


-.319 


1.730 


.084 


Man Months 


22 


-.350 


1.880 


.060 
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Whlle It appears that project leaders did give some 
consideration to time success, as they had evaluated It, In 
arriving at an overall rating of project success, that relation- 
ship, too, may reflect a "user satisfaction" bias. This last 
statement follows from the findings In Table 38, Time success, 
as viewed by the project leader, was related to reported project 
control, which has already been discussed; to the actual time 
success measure; and to the project leader's rating of cost 
success. However, It was also related to user satisfaction, 
as viewed by the project leader, perceived user participation, 
and the reported specificity of user requirements. These 
last mentioned relationships could reflect the tendency of 
project leaders to view time success In the context of how 
satisfied they felt users were with project results. This 
possibility Is supported by the fact that the two lowest 
ranked projects (19 and 20) on the user satisfaction criterion 
were rated at the lowest level (very poor) on time success 
(Var. 49) by the project leaders. However, these two projects 
were ranked 13 and 14 on the actual measure of time success 
(Var. 48). 

On the other hand. It may be that project leaders 
did rate overall project success with consideration given to 
their perceptions of both time success and user satisfaction. 

If this was the case, and the two projects cited in the previous 
paragraph were merely chance occurrences, the user satisfaction 
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relaclonshlp to perceived time success could be explained by 
the common relationship of both of those variables to overall 
project success (Var. 56). 

With respect to the strong relationship between the 
project leader's perceptions of time success and cost success, 

It would appear that a project leader response set was partially 
responsible for this relationship. Although, as Table 15 
shows, the actual measures of time success and cost success 
were related, that relationship was not significant at the 
.10 level. This led to the conclusion that project leaders 
tended to rate cost success and time success at about the 
same level, regardless of how similar the actual time 
success and cost success values were. 

Perhaps the most Important point to be made concerning 
the relationships shown In Tables 40 and 41 Is how accurately 
the project leaders perceived user satisfaction with their 
projects. It was evident that In nearly all cases, project 
leaders were very well attuned to how the users viewed project 
results. It was also apparent, as has been discussed above, 
that those user attitudes were key determinants In how 
successful project leaders viewed their projects to be. 

Different Views of Implementation Problems 

One Interesting aspect of project leaders' perceptions 
concerning user satisfaction was their awareness of Implementa- 
tion problems, as viewed by the users. As Tables 40 and 41 
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show, the users' evaluation of Implementation problems was very 
strongly related to how well satisfied the project leader 
perceived the users to be, and, in turn, to how successful the 
project leader felt the overall project was. It has already 
been shown (Table 22) that users considered ease or difficulty 
of project implementation to be a very significant component 
of overall satisfaction with a project. 

However, as Tables 42 and 43 show, project leaders 
viewed implementation problems very differently than users did. 
Not only were the two views of implementation problems different 
project leaders did not even consider implementation problems 
to be related to any criterion of success. 

Most of the relationships in Table 42 have been 
covered in earlier discussions of the several variables shown, 
so there is no need to repeat what has been said about them. 
However, taking all of the variables together, and considering 
the’ pattern, provides insight into what decreased implementation 
problems from the users' perspective. 

Implementation to users appears to have meant getting 
the project and its outputs Incorporated into their ongoing 
operations as smoothly and effectively as possible. The object- 
ive of the user was to get an Information system product which 
aided in decision making, but which also caused the least 
turbulence in the organization. Achieving this objective 
appeared to be tied to how prepared the user was for what he got 
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TABLE 42 

Implementation Problems -User (Var. 53) 

(Two-tailed significance at or beyond .10 only) 

Var. Tau Z(s) P(Z) 



User Satisfaction-User 


54 


.720 


4.192 


.0004 


Project Success -PL 


56 


.553 


2.920 


.004 


Combination Analyst/Prograniner 


23 


.530 


2.029 


.042 


User Satisfaction-PL 


57 


.519 


2.871 


.004 


Specificity of User 
Requirements -PL 


38 


.419 


2.290 


.021 


Mean Years Formal Education 


32 


.336 


1.884 


.060 


Quality of Documentation 


36 


.333 


1.739 


.082 


Originator 


11 


.330 


1.709 


.088 


User Participation -User 


46 


.291 


1.701 


.090 


Man Months 


22 


-.383 


2.217 


.026 


TABLE 43 








Implementation Problems- 


PL (Var. 


58) 




(Two-tailed significance at or 


beyond 


.10 only) 






Var. 


Tau 


Z(sl 


P(Z) 


Quality of Documentation 


36 


.533 


2.869 


.004 


Turnover 


29 


-.322 


2.327 


.020 


Complexity 


13 B 


-.352 


1.793 


.074 
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When he had originated a project, participated heavily in its 
development, and had good project documentation to work with, 
the user knew what was coming. Further, where he was able 
to deal with one individual who knew the whole picture when 
problems did arise, the user felt he was able to get those 
problems rectified quickly. 

Contrast this to how the project leader viewed imple- 
mentation problems. Although, as was mentioned earlier, the 
project leader was aware of the users* view of implementation 
problems, he did not consider the same factors to be the 
components of implementation problems, with the exception of 
documentation. The project leader tended to view successful 
implementation as getting the programs running, the documenta- 
tion finished, and the procedural flows correct so that his 
system worked. Turnover, although very limited in the sample, 
was seen as standing in the way of easy implementation, as was 
pressure to rush a project to completion,^ Reported complexity 
of the project was also negatively related to the ease of 
implementation, but it was difficult to determine which way 
this relationship went. It could be that project leaders 
viewed as complex those projects which were difficult to 

^Although not significant at the ,10 level, implementation 
problems (Var. 58) was negatively related to project control 
(Var, 42) with a tau of -,307 and a normal deviate of 1,683, 
significant at the .102 level (two-tailed). 
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implement. Finally, where documentation was felt by project 
leaders to be good, implementation problems were felt to be less. 

Although not entirely accurate, one generalization 
which might best describe the differing views of implementation 
problems is that project leaders tended to view implementation 
up to operational cutover of a project, while the users looked 
at implementation as what happened after operational cutover. 

Project Leader*s Perception of User Participation 

In general, the project leader felt there was high 
user participation if a project team consisting of some user 
representatives developed the project, and if he perceived top 
level user management had actively supported the project. 

Most of the relationships in Table 45 represent either these 
two basic variables or other variables already shown to be 
related to them. Further, the project leader tended to feel 
that the user had stated clear and specific project require- 
ments where there was a high level of user participation, as 
indicated by the fact that most of the variables shown in 
Tables 44 and 45 are the same. 

On the other hand, the users* evaluation of the 
specificity of their original desires and requirements was 
related to a separate group of variables as shown in Table 46. 
This evaluation was not significantly related to the project 
leader's view of ostensibly the same factor, specificity of 
user requirements. It appeared that users felt they had to 





!• 



.... . 




I « * ii* 




♦ •' 







i«- 




«»° flBMI 




• t f» 





l> 





^ «■ •••I H 



I 



Ml 







• «•• 




M 











- 155 - 



be more specific In stating what they wanted In larger organiza- 
tions to get their projects accepted. It also appeared that 
the users were more accurate than project leaders in their 
appraisals of how clearly they stated exactly what they wanted 
from projects. Although not significant at the .10 level, 
the users' view of specificity of their requirements (Var. 45) 
was related to the time success criterion (Var. 48) with a tau 
of -.272, a normal deviate of 1.586, and a probability of .114. 
Further, there was a significant relationship between Variable 
45 and both the elapsed months spent on the project, and the 
project leader's appraisal of time success. 

TABLE 44 

Specificity of User Requirements -PL (Var. 38) 

(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


Z(si 


P(Z) 


Project Team 


14 


.750 


3.032 


.0027 


User Man Months/ 
Total Man Months 


15 


.562 


3.073 


.0022 


User Participation -PL 


41 


.562 


3.036 


.0025 


Implementation Problems-User 


53 


.419 


2.290 


.021 


User Sat is fact ion -PL 


57 


B.409 


2.116 


.036 


User Participation-User 


46 


.400 


2.160 


.030 


Project Success-PL 


56 


.367 


1.937 


.052 


User Management Interest 


39 


.320 


1.675 


.094 


Two or More Years Systems 
Experience 


27 


-.450 


2.435 


.014 
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TABLE 45 

User Participation -PL (Var. 41) 
(Two-tailed significance at or beyond .10 only) 





Var. 


Tau 


..ZCs)_ 


P(Z) 


Project Team 


14 


.660 


2.603 


.009 


Specificity of User 
Requirements -PL 


38 


.562 


3.036 


.0025 


User Man Months/ 
Total Man Months 


15 


.539 


3.233 


.0014 


User Management Interest 


39 


.453 


2.332 


.020 


User Satis faction-PL 


57 


.394 


2.135 


.032 


User Participation -User 


46 


.367 


2.174 


.030 


Measurable Project Objectives 


12 


.344 


1.830 


.068 


Two or More Years Systems 
Experience 


27 


-.361 


2.141 


.032 


TABLE 46 








Specificity of User Requirements -User 


(Var. 45) 




(Two-tailed significant 


;e at 


or beyond 


. 10 only) 






Var. 


Tau 


. ZCs) _ 


PCZ) 


Employees 


3 


B.405 


2.316 


.020 


Time Success -PL 


49 


.394 


2.114 


.034 


User Participation -User 


46 


.356 


2.111 


.034 


Elapsed Months 


21 


-.300 


1.756 


.080 


College Degree 


5 


-.333 


1.977 


.048 
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On the assumption that clear, specific, and detailed 
user requirements provided the project staff with a more concrete 
nucleus around which to build the project, time success should 
have been greater. The data would appear to support this 
contention. 

User participation (Var. 46) also appeared to be 
greater when users felt they had been very specific in stating 
their requirements. As was mentioned at an earlier point, 
where the users were able to state exactly what they wanted from 
a project, they apparently felt they had control of what was 
developed, and, therefore, that they had high participation. 

It may strike the reader as odd that there was no 
relationship between reported specificity of user requirements 
(Var. 45) and user satisfaction (Var. 54). However, a close 
investigation revealed three projects, termed "evolutionary", 
which were low on Variable 45, but high on user satisfaction 
(Var. 54). There were also three projects which were high on 
Variable 45, but very low on Variable 54, a result, apparently, 
of users not getting what they had specified as required. 

These several projects, in effect, "cancelled out" any relation- 
ship between specificity of user requirements (Var. 45) and 
user satisfaction (Var. 54) in the computation of the tau 
statistic. 

In summary, whereas the project leader appeared to 
base his evaluation of the specificity of user requirements 



158 - 



on the degree of user participation in the project, the users 
seemed to base the appraisal of their participation, in part, 
on how specifically they stated their requirements in the first 
place. There were cases, however, where users were not specific 
in stating their requirements, but felt they participated 
through an evolutionary process of trial and error over a 
relatively long period of project development. Finally, users 
and project leaders did not tend to agree on the degree of 
specificity in original project requirements, but the data 
suggest that perhaps the users' appraisals were more accurate 
than those of project leaders. This last point, on the accuracy 
of appraisals, should be researched more thoroughly, however, 
before any firm conclusion is drawn. 

Quality of Documentation 

For the projects in the study, the perceived quality 
of documentation was not related to the presence or absence of 
documentation standards. Again, it is emphasized that only 
four projects were developed without documentation standards, 
so it would seem unreasonable to conclude that documentation 
tends to be as good without standards as with them. However, 
it did appear that the mere presence of standards was not enough 
to assure high quality documentation. 

As shown in Table 47, the perceived quality of project 
documentation wac velated to several other variables. It has 
already been poin» ^d out that both users and project leaders 
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percelved fewer Implementation problems where documentation was 
of relatively high quality. This finding was not unexpected. 

TABLE 47 

Quality of Documentation (Var, 36) 

(Two-tailed significance at or beyond .10 only) 



Var. Tau Z(s) P(Z) 



Implementation Problems-PL 


58 


.533 


2.869 


.004 


Implementation Problems-User 


53 


.333 


1.739 


.082 


High Level Progranning 
Language 


43 


.300 


1.780 


.076 


Man Months 


22 


-.327 


1.663 


.096 


Project Control 


42 


B-.336 


1.650 


.10 


Elapsed Months 


21 


-.340 


1.736 


.082 



The relationship of perceived documentation quality 
to the use of higher level programming languages, essentially 
COBOL, has also been discussed. It appeared that most project 
leaders who used COBOL felt they had adequate documentation to 
allow reasonably easy program maintenance, which seems to have 
been their primary criterion for quality. 

Finally, the perceived quality of project docvtmenta- 
tion was negatively related to the elapsed time on the project, 
the man months on the project, and reported project control. 
These rather weak relationships could be interpreted at least 
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two different ways. One Interpretation would be that projects 
took longer because of poor quality documentation that occasioned 
delays, duplication, and general confusion. Another interpreta- 
tion, which seems to make some sense, would be that where project 
control schemes were more stringent, pressure built up on project 
leaders and staffs as the project took longer or more man 
months were expended. The response to the pressure was to cut 
corners on documentation, concentrating on getting something 
running as quickly as possible. If the latter possibility were 
true, the control schemes used could have been dysfunctional 
to long range project success. Most of the prescriptive litera- 
ture argues for both documentation standards, to assure high 
quality documentation, and project control techniques, to keep 
projects on target. They may not be compatible. This is a 
very interesting question which should be investigated by 
experimental means if possible. 



PART IV 

SUMMARY OF RESULTS, CONCLUSIONS, AND 



SUGGESTED FUTURE RESEARCH 
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CHAPTER VIII 
SUMMARY OF RESULTS 

Introduction 

In Chapter I the purpose of the research described In 
this thesis was phrased as the question: 

What organizational and procedural factors 

are correlates of success with MIS projects? 

In seeking answers to that question, the literature on MIS, 
project management, and Information systems In general was 
reviewed. While few satisfactory answers were found In the 
literature review, a set of thirty-five hypotheses was 
developed which. It was hoped, could be tested. 

The list of hypotheses was pared down from thirty-five 
to sixteen after analyzing the responses of Information systems 
professionals to a request to rank the thirty-five hypotheses, 
or factors. In terms of their Importance to MIS project success. 
These sixteen hypotheses, along with four criteria of MIS project 
success, then became the nucleus of a questionnaire which was 
developed for Interviewing people In organizations who had been 
Involved with MIS projects. 

A successful pre-test of the questionnaire was followed 
by the selection of a study sample. Ten organizations from the 
Twin Cities, representing seven Industries, were chosen for the 
study. Data were collected on two MIS projects In each of these 
organizations between October and mid-December, 1970. 
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The data collected were analyzed to determine what 
relationships existed among variables In the study. The primary 
form of statistical analysis was the computation of Kendall's 
rank correlation statistic, tau, for every possible relationship 
between any two of the sixty -one variables Identified in the 
study. 

The results of the data analysis have been discussed 
In detail In Chapters VI and VII. This chapter Is devoted to 
suimarlzlng the findings of the study. The final chapter. 

Chapter IX, will be devoted to drawing conclusions from the 
results, and to suggesting some possible future research 
directions based on what has been found here. 

Independence of Criteria of Success 
Four criteria of MIS project success were posited and 
used In this study: time success, cost success, user satisfac- 

tion, and computer operations success. These four criteria were 
found to be reasonably Independent measures of MIS project 
success. The closest relationship found between any two criteria 
was that between user satisfaction and computer operations success. 
This would seem to be a logical relationship. Where a project 
Impacts on computer operations In such a way that scheduling 
and running problems develop, the users probably feel such 
problems through Inadequate response and service from the 
Information systems function. 
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Findings; Testa of Hypotheses 
Hypotheses Related to Criteria of Success 

Of the sixteen hypotheses which were selected to be 
tested In the present study, one was not effectively tested 
by the sample data, ten were found to be significantly related 
to at least one criterion of success, and five were found to 
be unrelated to any criterion of success. The hypothesis 
which was not tested, and about which no conclusions could be 
drawn, was the degree of relationship between centralization 
of the information systems function and MIS project success. 

The ten hypotheses which were tested and found to 
be related to project success are shown in Table 48. For 
each entry, the rank of the hypothesis, in terms of its importance 
to project success, from the original ranking of thirty-five 
hypotheses is shown first; next is shown the hypothesis itself; 
and shown last are the separate criteria to which the hypothesized 
factor was related. A in the table indicates that the 
factor contributed to success on the criterion in question, 
and a indicates that the factor detracted from success on 
the criterion in question. Tables 16-19 in Chapter VI should 
be consulted to find the strengths of the relationships shown 
in Table 48, and to determine the variable numbers used in the 
study to represent the hypotheses. Further, the discussions 
in Chapter VI concerning Tables 16-19 should be referenced for 
interpretation of the relationships. 
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TABLE 48 

Summary of Findings 



User 



Rank Hypothesis 


Time 


Cost 


Satis* Computer 

faction Operations 


1 


Participation by operat * 
Ing management In design » 
formal approval of 
specifications, and 
continual review of 
project. 










7 


Organization level of 
top computer executive. 


— 






+ 


12 


Documentation standards 
used and enforced. 


+ 








14 


Low turnover of project 
personnel. 






+ 




16 


Source of origination of 
project (MIS staff or 
user) . 






+ 




20 


Length of experience In 
the organization of 
project personnel. 






+ 




23 


High level programming 
language used for 
project. 


+ 








31 


High formal educational 
level of project 
personnel. 


“ 








32 


Separation of analysts 
and programmers for 
large projects. 










33 


Overall size of organl* 
zatlon systems staff. 


— 




+ 
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Hypotheses Not Related to Criteria of Success 

The £ive hypotheses found to be unrelated to project 
success for the projects In the sample were: 

Rank Hypothes is 

2 Measurable project objectives from 
conception of the project. 

3 Utilization of a project team composed 
of MIS staff and user personnel. 

8 Systems experience of project personnel. 

13 Use of a formalized and regular reporting 

structure on project progress. 

34 Ratio of computer hardware Investment to 

total sales or operating budget. 

General Comments Concerning the Hypotheses 
In reviewing the above findings from the study, one 
Impression that Immediately comes to the fore Is the error In 
the evaluations of the Information systems professionals who 
ranked the thirty-five hypotheses In terms of contribution to 
project success. More of the hypotheses were confirmed from 
the lower half of the ranking than from the top half. In other 
words, those factors which the Information systems professionals 
thought most crucial to MIS project success tended to be rejected 
more than those factors thought least crucial. However, the 
questions and possible explanations arising from this situation 
are more Interesting than the situation Itself, 

A fundamental benefit to the researcher In conducting 
this study was that the general confusion which surrounds the 



r 




167 



definition of MIS was brought into clearer focus. That there 
is rather widespread disagreement among those in the informa* 
tion systems community concerning what MIS really is, or ought 
to be, became apparent in analyzing the responses of the 
information systems professionals who attended the Founding 
Conference of Society for Management Information Systems in 
1969 to a request to define what MIS meant to them. The 
different definitions of MIS were almost as numerous as the 
respondents themselves (Dickson, 1970). 

The researcher tried to avoid this confusion from 
the beginning of this study by concentrating on MIS projects, 
and by defining rather explicitly the criterion which placed 
a project in the management information system category. Very 
simply, any project selected for study had to result in infor- 
mation being provided to a manager, at some level, which was 
used by that manager in the decisionmaking process relative 
to his domain of responsibility. The emphasis throughout the 
study was on looking at projects which had been conceived to 
furnish managers Information that would assist them in making 
decisions in a more effective way. 

The conclusion reached during the course of the 
research was that there are at least three different types of 
computer-oriented Information system projects which should not 
be lumped together. These three are; 
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!• Data processing projects, where the basic 
objective is to process operational data in 
a more efficient manner by using a computer 
in lieu of manual or accounting machine methods. 

2. Generalized software projects, where the object- 
ive is to produce a marketable software package 
for widespread consumption by various users. 

There is no explicit concern here for the specific 
information needs of one or a few users in one 
organization, although the products of such an 
endeavor might facilitate responding to those 
needs after further development, modification, 

or incorporation into a user -oriented system. 

3. Management information system projects, where 
the objective is to provide a manager, or 
managers, with specific information system 
products that will enable those managers to do 
a better job of decision-making relative to 
specific kinds of problems. 

As has already been pointed out, this research has 
focused entirely on category 3 above. However, the specification 
of a definition for the purposes of this study can in no way be 
expected to go beyond the bounds of this study. In other words, 
although a definition of MIS projects has been provided which 
has suited the purposes here very well, it is only one of many 
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posslble definitions. The data and results presented In this 
thesis are relative to only one portion of the Information systems 
spectrum, and where factors were ranked by others In the field 
as to their importance to MIS project success, other frames of 
reference were possible. Therefore, to say that the conclusions 
reached here serve to reject the general thinking of professionals 
In the Information systems field as to what factors are critical 
to MIS project success would be an error. 

With that background, several further coninents can be 
made about the findings and conclusions presented here. Since 
the entire focus of the present study has been MIS projects, 
as they have been defined, all of the findings must be viewed 
In that context. The size and scope of projects, how they 
were managed, the Interactions that occurred, and measures of 
success must all be considered In the context of the MIS project. 
This means that generalizations beyond this type of project 
should only be made with the greatest caution. If at all. This 
Is not to say that the findings here are of no consequence because 
they don't apply to the entire Information systems spectrum. 
Rather, other portions of that spectrum must be Investigated 
separately, or at least with an awareness that there seem to 
be fundamental differences In what one Is looking at. However, 
even this last statement cannot be confirmed until these other 
segments of the Information systems spectrum have been analyzed 
In somewhat the same way that MIS projects have been analyzed 
In this study. 
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The User Participation Cluster 

It was definitely concluded from the analysis of 
the data collected in the study that perceived user participa- 
tion is a crucial factor in the success of MIS projects. There 
was a cluster of variables that seemed to represent, essentially, 
the direct and close involvement of users in the development of 
their projects. This cluster consisted of origin of the 
project, reported clarity of objectives, reported specificity 
of actual Information requirements, the use of an in ter functional 
project team, and finally, the user participation variable 
itself. However, making this general statement necessitates 
explanation of several points with respect to success and the 
separate variables just cited. 

First, there is an implicit value judgment that user 
satisfaction with a MIS project is the most important success 
criterion. This is based on the nature of the MIS project as 
opposed to other possible project types. Since the primary 
purpose of the MIS project is to provide information to a 
manager to support his decision-making responsibilities, the 
failure of a project to satisfactorily serve this end means 
failure for the project in terms of its essential purpose. 

Being within budget on time and cost, and creating no problems 
for computer operations, are hardly meaningful if the completed 
project is not used or is viewed by users to be inadequate 
for their information needs. This is not an advocacy of 
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complete freedom from budgets or the need to consider how a 
given project will affect computer operations. It is merely 
a value hierarchy of the criteria, with user satisfaction as 
the primary criterion and the other three as Important, but 
secondary criteria of success. 

Moving to a closer consideration of the variables 
included in the cluster mentioned above, several important 
aspects of perceived user participation were brought out in 
the analysis of sample data. First, although clear and measur- 
able project objectives would logically seem to be a big 
factor in users getting what they could use, and within time 
and cost expenditures reasonably close to budgets, this was 
not always the case. In fact, there were enough projects in 
the study which did not follow this logical pattern that the 
nature of project objectives was not significantly related to 
any criterion of success. Those projects which did not fit 
the expected pattern were of two types: projects of the "evolu- 

tionary" type, and projects where users knew what they wanted 
to be able to do, but did not understand their own information- 
decision environments well enough to know how to do it. 

In the first type, the "evolutionary" project, 
objectives were often reported to have been rather vague at 
the beginning of a project, but the users worked so closely 
with the project staff that, over a fairly long period of time, 
during which the users progressively learned more about what 
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Informatlon they could use and in what ways, results were 
generated with which the users were highly pleased. In the 
second type, where it was reported that there were clear and 
measurable project objectives, but where the users were not 
able to specify precisely what information they needed, or 
in what form, to reach those objectives, a good deal of 
frustration and slippage occurred during project development. 

It would seem that in these cases a project was begun before 
enough spadework had been done by users. In an effort to 
turn something out the project staff, after floundering for 
a length of time, froze requirements which may or may not have 
been what the users ended up feeling they should have gotten 
from the project. 

With respect to the source or originator of a project, 
those initiated by users were generally more successful. The 
users usually knew what they wanted, and were thus able to work 
with project staffs in getting it. However, where users initiated 
a project, but then did not participate very much in the develop- 
ment efforts, results were poor. While the lack of user partici- 
pation may have resulted, in part, because the project leader 
did not allow it, the major problem appeared to be users who 
were unwilling to contribute in a meaningful way. As a conse- 
quence, the project leader made decisions by default which 
affected the acceptability of the final products in the eyes of 



the users. 
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From the preceding conments one draws the conclusion 
that some organizations, and some individuals in organizations, 
have not come to recognize a difference between MIS projects and 
data processing projects. This situation was nowhere more 
apparent than in the findings regarding the use of in ter functional 
project teams. While teams did appear to enhance user partici- 
pation, there were cases where the team approach was implemented 
in such a way that project success was marginal, if not poor. 

There were enough of these cases that project success and the 
use of teams were not significantly related to each other. 

It was found that merely setting up a project team 
with users represented does not insure participation of the 
ultimate users of project outputs. In several cases, the data 
processing orientation seemed to dominate. It was assumed by 
user management, erroneously, that merely having someone on the 
project staff from the user area meant there would be high parti- 
cipation, and the user management would end up with satisfactory 
results. This approach may have served quite well for data 
processing projects, where the primary focus was on getting 
procedural aspects of an operational function cut over to the 
computer. A staff specialist from the user area who knew the 
procedures, records, and so forth, could do an excellent Job 
for the user function in working as a member of the data 
processing project staff. However, where the project is 
primarily oriented to providing a manager with information to 
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ass 1st his decision making, that manager, not the staff 
specialist, should be the one involved. The fact that in 
some cases this did not occur gave rise to the disagreement 
found among user personnel interviewed about the same project. 
The staff person from the user area felt he had participated 
heavily, and the results of the project were very good, whereas 
a manager, who was actually receiving the project outputs, felt 
participation was not so high and the results were not so good. 

Another problem, related to the one just discussed, 
was the case of the projects where the user representatives who 
worked on the project staff were not the people who actually 
implemented the project. For one reason or another, the user 
representative was given other tasks, not directly related to 
the MIS project studied, soon after the project had been 
completed. This meant that people unfamiliar with what had 
transpired during project development were forced to implement 
the project, and to maintain continuing liaison with the 
information systems function. The results, not unexpectedly, 
were rather poor. The situation just described occurred either 
because the user representatives on the project team were staff 
personnel who were put on something else as soon as the project 
was completed, or the managers who had wanted the project 
developed, and worked with the project team, were transferred 
shortly before or after project implementation. 
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The Project Management Cluster 

Three variables were viewed as comprising a project 
management cluster: documentation, project control, and the 

choice between separate or combination analys t /programmers • 

All three of these factors have been subjects of considerable 
discussion in the information systems literature, as pointed 
out in Chapter II, It would appear that much of the clamor 
surrounding these three factors has arisen as a result of 
the difficulties experienced with the development of data 
processing projects and generalized software projects. However, 
no distinction has generally been made among the different 
kinds of project environments in prescribing what those 
involved in developing computer based systems should do. The 
implicit assumption seems to have been that all projects are 
essentially the same, and, therefore, the same nostrums which 
were believed to be useful in remedying data processing and 
generalized software development ills would be applicable in 
the management information system environment. 

There was certainly no evidence in this study which 
leads one to advocate anarchy in the management of MIS projects; 
which nudges one toward the conclusion that project management 
efforts are futile and should be abandoned. However, there 
was evidence to support the position that MIS project manage- 
ment should be considered in the MIS context. Unfortunately, 
in this regard, about the best the data analysis from this 
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study can do is raise some questions which need to be investi- 
gated thoroughly. 

What are these questions that analysis of the data 
collected have brought to focus? First, is the issue of 
assigning separate analysts and programmers to a MIS project 
versus assigning one person who performs both functions. This, 
of course, does not mean that only one person would be assigned 
to a project in all cases; rather, that the project would be 
divided into functional modules where one individual takes 
responsibility for all analysis and programming involved for 
assigned modules* 

Over the past several years, the sentiment has 
shifted from time to time as to which approach should be used; 
at no time has there been general agreement on which approach 
is best. There have been advocates in both camps. It would 
seem that this issue, as with the others raised in this chapter, 
is one without a universal solution. What is "best** probably 
depends on the organization and information system environment. 

At least for the MIS environments of organizations 
sampled in connection with the research reported here, the use 
of combination analyst /programmers tended to yield the best 
results in terms of user satisfaction with project products. 

This seemed to stem mainly from the ability of the combination 
analyst/programmer to respond more quickly and effectively to 
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user requirements and problems. Where the user was able to 
deal with one person who had the whole picture with respect 
to one or more functional modules of a project, the user 
appeared to be more confident of what he could expect from 
the Information systems staff and tended to be less 
frustrated by the Implementation problems that arose. 

To say that all MIS projects should be developed 
by combination analys t/programmers would, of course, be an 
unreasonable conclusion to reach. Several organizations had. 

In effect, a dual system, where some combination analyst/ 
programmers were used, but where newer personnel, trainees, 
were also used as programmers only. This approach seems to 
have some merit. In that new people must be trained by some 
means. The question that should be raised, however. Is whether 
or not this Is the most effective means of training new systems 
people* This Issue should be Investigated, because It Is 
possible that user satisfaction with MIS project results may 
be adversely affected where the separate analys t/programmer 
approach Is utilized as a means of training new people. The 
Important point Is for the management of the Information systems 
function to be aware of this possibility, so that safeguards 
can be built Into project management to minimize adverse Impact 
on user satisfaction with project results. 

A second aspect of MIS project management which would 



seem to require a reappraisal Is project control techniques. 
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For the projects in the sample, the project control variable 
was not related to any criterion of success, and it was negatively 
related to the project leader*s appraisal of documentation quality. 
These findings could be interpreted in several ways. However, an 
explanation that encompasses all of these findings on project 
control is simply that the methods of project control were not 
conceived nor applied in the MIS context. 

Where stringent progress reporting against planned 
activities was required, that reporting did not, in general, 
result in actual control being exercised. In the case of only 
one project were additional resources committed to enable the 
project staff to achieve the target date for completion. In the 
other cases, progress information seems to have been collected 
for its own sake, with little apparent use made of the information 
for control purposes. Where target dates were revised backwards 
as a result of slippage, this was of value to those in the organi- 
zation who were to be affected by the completed project, but 
such use of supposed control information would not seem to 
constitute the whole realm of project control. 

On the negative side, where stringent project status re- 
porting was required, project staffs seemed to have felt pressure 
to get programs running as quickly as possible. However, where 
they had no additional resources available to them, about all 
project staffs could do to ease the pressure was cut corners 
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on documentation and implementation preparations, and refuse 
to make changes requested by users. In other words, it 
appeared that project staffs were frustrated by control methods 
rather than helped by them. 

The reader may feel that the application of pressure 
as a means of reaching target dates is useful, nonetheless; 
that without such reporting and pressure, projects could go 
on for longer periods than they already do. And it is 
certainly true that with respect to the projects in this 
sample which were subject to tight progress reporting require- 
ments, the actual times may have been less than they would 
have been had there existed no progress reporting. It is 
impossible to resolve that question, of course. The really 
important point in all of this, however, is the relation of 
the pressure to the quality of the original project estimate. 

It appeared to the researcher that project time and 
cost estimates were probably made relative to data processing 
projects rather than to MIS projects. In very few cases was 
there recognition at the beginning of a project that, in the 
MIS environment, managers must learn to use what they receive; 
that they may want to change the content or format of outputs 
as they go along to whatever is easiest for them to use or 
fits into their decision-making styles. Therefore, as a result 
of poor estimates in the first place, some project staffs were 
subject to pressure to complete projects within artificial time 



frames . 
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Finally, with respect to perceived quality of project 
documentation, and documentation standards, no significant rela- 
tionship was found between these two variables. Although 807. of 
the projects in the sample were subject to some form of documen- 
tation standards, the perceived quality of documentation was not 
assured by such standards. It should be pointed out that a weak- 
ness of the present study was the absence of any objective measure 
of project documentation which would allow the comparison of all 
projects against the same specifications. However, the evaluation 
of project documentation by project leaders was valid to the extent 
that projects free of documentation problems did seem to meet a 
functional test of documentation effectiveness. The assumption 
was made that where there were no documentation problems perceived 
by those who were using the documentation, the documentation met 
the functional test of quality. 

The above conclusions are not Intended as a case for 
no standards or no control with MIS projects. Rather, the 
point to be made is that both control and documentation standards 
should be conceived and applied to MIS projects with an explicit 
recognition that the MIS project is different from the data 
processing project or the generalized software project. 
Documentation standards should be oriented to providing clear 
understanding among all those concerned with a project, rather 
than to fixing responsibility for who said what to whom. The 
whole thrust of the standards should be toward making it easier 
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for evolution with users to occur rather than inhibiting it. 

The standards should be such that change is facilitated, rather 
than prevented through freezing requirements too rigidly at 
some arbitrary point. Project documentation is certainly no 
less important because the project is oriented to providing 
information to managers to assist their decision-making. 

This was borne out by the relationships between the perceived 
quality of project documentation and both users' and project 
leaders' views of implementation problems. However, the 
kinds of doctmientation standards which have been applied do 
not appear to be the most effective for assuring the type 
of documentation needed with MIS projects. 

The same sort of argument holds for project control 
methods. All control should not go out the window because the 
MIS project is somehow different from the kinds of projects 
control schemes have been devised to handle. In fact, it is 
quite possible that MIS project control will provide a 
greater challenge to information systems managers than data 
processing projects have. Where time estimates cannot be 
fixed as easily, and where several projects are running at 
one time in various stages, the allocation and control of 
resources will likely be more difficult. But the point is, 
that if managers are to derive truly usable products from 
MIS projects, they should be allowed to grow and experiment 
with results. This means a more fluid situation where estimates 
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are not based on just the apparent facets of a project at 
original definition. The estimates will also have to be based 
on the kinds of products Involved and the individual situations 
of the managers who are to be served. The notion of many open- 
ended projects, current over a long period of time, may seem 
to be an Impossible situation with which to live. But what 
is the point of imposing artificial constraints from another 
environment on MIS projects, only to achieve time and cost 
^success** for results that are ineffective or not used? 

The Project Leader^s View of Project Success 

It has been argued here that the most crucial criter- 
ion of MIS project success is user satisfaction with the 
results of the project. How do project leaders view project 
success? It would appear, based on the sample data, that 
project leaders Implicitly, if not explicitly, understood the 
ascendency of user satisfaction over other criteria for MIS 
projects. Project leaders* perceptions of project success 
were related very strongly to how satisfied users were with 
project results. Further, project leaders clearly recognized 
the great importance users attached to implementation problems. 
Where users were able to use project outputs with relative 
ease, project leaders felt the projects were successful. Most 
project leaders also seemed to recognize that cluster of user 
participation variables, so important to user satisfaction, 
as important in contributing to project success from their own 
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viewpolnts . 

While project leaders did accord time success some 
importance in evaluating overall project success, this was 
secondary to user satisfaction. Further, the project leader's 
view of project success was not significantly related to either 
cost success or computer operations success. In short, project 
leaders were keenly aware, in most cases, of how the users felt 
about projects, and these users attitudes were the most impor- 
tant key in how successful the project leaders felt the projects 
were. 

Differing Views of Users and Project Leaders 

Although users and project leaders agreed very well 
on what made MIS projects successful, they disagreed on two 
important aspects of MIS projects. Project leaders appeared 
to consider implementation problems as those difficulties 
encountered in getting a project completed and operational. 

They tended to focus on cutover as the crucial point, and 
where problems arose in making the cutover, they felt there 
were implementation problems. Users, on the other hand, seemed 
to focus on problems in using the project products as implemen- 
tation problems. They tended to look at implementation as 
what occurred after cutover; those problems experienced in try- 
ing to work with what had been developed to assist them in 
their decision-making roles. 
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Thls finding may appear to be a contradiction of 
what was said earlier concerning the project leader's percep- 
tions of success, and how those perceptions were strongly 
related to users reporting few implementation problems. However, 
there really is no contradiction. The project leader, nurtured 
in the data processing environment, tended to look upon imple- 
mentation as getting a set of programs up and running. Once 
this was accomplished, and programs were reasonably free of 
obvious bugs, he and his staff went on to something else, 
new projects. Users, on the other hand, had to live with 
what had been given them. No matter how free of technical 
bugs a program was, users had to try to use what they received. 
Where there was no follow-through by the systems staff in 
helping users to do this, users became frustrated and felt 
there were implementation problems. Although project leaders 
seemed to be aware of the frustrations and problems felt by 
users, and recognized these frustrations and problems as 
important factors in how well satisfied the users were, the 
project staffs were often already comnitted to other projects. 

This meant that although they recognized that follow-through, 
and perhaps changes , were necessary to help users get the kinds 
of information needed, they were not free to do anything about it. 

A second area where users and project leaders disagreed 
to some extent was in evaluating how specific original user 
requirements had been. The project leaders tended to lump 
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specificity of user requirements in with the cluster of user 
participation variables, whereas users appeared to evaluate, 
in retrospect, how clearly they were able to state what they 
wanted at the beginning of the project. 

The evaluations by users of how specifically they 
had stated what they wanted were probably colored by the 
growth process in working with outputs. It appeared that 
users tended to be less satisfied after a period of working 
with a project than when the project was just completed. This 
was interpreted as a reflection of user learning. Thus, when 
the user was interviewed, his feeling that he was not presently 
getting what he would really like to have probably Influenced 
his judgment of how clearly he had stated what he wanted in the 
first place. He seemed to assume that if his requirements 
were not now being satisfied as well as he would like, he had 
done a poor job of saying what he wanted. This, of course, 
was not necessarily the case, because the user may have stated 
fairly well what he wanted when he started. But through the 
experience of using project outputs to do his job, his require- 
ments had shifted to some degree. 

The project leader, from his perspective, was 
concerned with trying to pin requirements down so something 
could get programmed. The fact that he had users working with 
him on a team, or readily available to work with his staff if 
not on a team, probably facilitated his getting specific 
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requirements defined. The fact that the project leader sought 
to pin down requirements was natural; It was his job to do so. 
However, the error was In assuming those requirements would not 
shift after the project was Implemented. It should not be 
construed that this was necessarily the project leader's error. 
The very nature of most data processing environments precluded 
his going very far beyond Implementation. The evolutionary 
approach was not recognized nor built Into the development 
process . 
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CHAPTER IX 

GENERAL CONCLUSIONS AND SUGGESTIONS 
FOR FUTURE RESEARCH 

General Conclusions 

The most fundamental conclusion drawn as a result 
of conducting the research reported here Is that different 
environments exist for different kinds of Information system 
projects. While this may sound like a truism, and a very 
superficial result of an extensive research effort, the Impli- 
cations of that conclusion pervade virtually all of the findings. 
If professionals In the Information systems field know that 
different kinds of projects do exist, and that they should be 
treated In different ways, that knowledge Is not readily 
apparent In either the literature of Information systems or 
the practice of those whose vocation Is Information systems. 

It was stated In Chapter VIII that at least three 
different types of Information system projects appear to exist. 
These three types of projects would all seem to place different 
kinds of requirements on those responsible for managing their 
development. The present study was devoted solely to MIS 
projects, so conclusions should not be generalized to other 
types. 

With MIS projects, the primary objective Is to provide 



Information to managers which they can use effectively In 
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carrylng out their decision-making responsibilities. For this 
reason, the satisfaction of the manager with the results of 
the MIS project should be viewed as the most Important criterion 
of project success. Time success, cost success, and computer 
operations success, while important, are secondary to user 
satis faction. 

To achieve success with MIS projects, the findings 
in this study suggest the following approaches to managing 
project development: 

1. Adopt the evolutionary concept of project 
development. This implies recognition of 
a learning process among users which may 
only begin, but is certainly continued, 
after the project has been initially cut 
over and the user is receiving outputs. 

Where the main objective of a project is 
to enable a decision-maker to function 
more effectively, the decision-maker must 
be given the greatest precedence in 
project development plans. Projects should 
be considered from their conception as rather 
fluid, with provision for a trial and error 
process of growth on the part of the user. 

The project should not be viewed, in most 
cases, as a one-shot development effort 



which has a specific end point. 

2. Because of the nature of the MIS project, 
follow-through by the information systems 
staff is imperative. Close Involvement 
with the user should not terminate upon 
initial cutover, but should continue to 
be supported by systems staff resources 
over a fairly long period of time. It 
would be expected that this resource 
commitment would decline as the user 
gains experience and confidence with the 
project outputs. Changes to project 
outputs would tend to be greater at first, 
decreasing in scope as the manager sights* 
in on what is really the most valuable 
information to him. 

3. Project management should be oriented to 
the requirements of MIS projects, as 
opposed to attempting to manage all informa- 
tion systems projects the same way on the 
implicit assumption all types are essentially 
alike. Documentation standards and project 
control techniques should be devised which 
allow evolution rather than thwart it. Time 
and cost estimates should be made in light of 



the evolutionary concept, and should not 
be used as cudgels to speed program convert* 
sion to free resources for other purposes, 

4, Wherever possible, combination analyst/ 
prograniners should be utilized to facilitate 
user satisfaction. The ability of the user 

to look to one person who can respond knowledg 
ably and quickly to his problems is impor- 
tant, and it would appear that the combination 
analyst/progranmer is better able to do this 
than individuals who have done only analysis 
or progranining work on the project, 

5, Large projects which cover several functional 
areas should be avoided. Such projects would 
seem to be too inflexible for MIS, and too 
far removed from individual users, Blumenthal 
(1969) framework for MIS projects would seem 
to be an appropriate model to follow; large 
systems could be broken into small modules, 
which are both more manageable, and more 
meaningful for users. The large underlying 
projects which are needed to support MIS 
projects could be approached as either data 
processing projects or generalized software 
projects. An example of the latter would be 
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a generalized file management system which 
can support many different MIS projects. 

6. User participation means just that; those 
who are participants in MIS project develop- 
ment should be the managers who will use the 
products. The technique of assigning a 
user area staff analyst to the project team 
was fine for data processing projects where 
the primary requirement was for user area 
procedural knowledge and expertise. However, 
the MIS project focuses on the manager and 
his information needs. The definition of 
those needs should not be left to someone 
else if the manager is to be satisfied with 
what he gets. 

The author has no delusions concerning the difficulty 
of following the above suggestions. No specific steps have been 
outlined which make it easy to implement any of these things. 
And, there is no guarantee that the separate suggestions, if 
followed, will bring success. However, the important findings 
of this exploratory study are that organizations should move 
towards different sets of ground rules for MIS projects than 
have been prescribed for data processing projects or generalized 
software projects. The objective of user satisfaction should 
guide the thinking of information systems managers as they seek 
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more effective ways to use the computer In their organizations. 

Suggestions for Future Research 
It Is believed that this study has pointed up several 
areas requiring more explicit and intensive research; areas 
which, if researched, should provide some answers to the thorny 
problems Inherent in pursuing the course that has been outlined 
above, ^song the Issues raised here which should be investigated 
thoroughly, and by experimental means, if possible, are: 

1. The kinds of Information used by managers in 
making decisions concerning their domains of 
responsibility, and the places they get such 
information now. 

2. The effects of various project control schemes 
on project staffs; the tradeoffs among maintain- 
ing order, dysfunctional pressure, and willingness 
to adapt to user desires which may put the project 
in conflict with the control scheme. 

3. Related to 2 above would be research on the 
reward structures in various organizations 
as they affect systems personnel engaged in 
developing MIS projects. What kinds of organi- 
zational rewards are tied to explicit and 
implicit organizational objectives? 

A. Techniques for allocation and management of 
information systems staff resources where 
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pro jects do not "end" at an Initial, clear 
cutover point. This would encompass techniques 
for estimating project time In the evolutionary 
approach . 

5. Techniques for Integrating managers Into the 

development efforts on projects Intended to 

provide them with Information for decision- 
1 

making. What different approaches are possible, 
and most effective. In getting managers to 
accept the kind of communication required In 
following the evolutionary concept of MIS 
project development? 

6. If combination analyst/programmers should be 
used to develop MIS projects, how might new 
systems personnel best be trained to minimize 
deleterious effects on user satisfaction? 

7. What are correlates of project success for other 
kinds of Information system projects, such as 
data processing projects and generalized soft- 
ware projects? In these other project environ- 
ments, what should be the criteria of success, 
and how should those criteria be viewed In terms 
of Importance? 

8. What kinds of project documentation are most 
crucial to MIS project success? Is It possible 
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to devise measures of documentation 
quality which can be applied objectively 
to all MIS projects studied to allow 
comparisons across projects and organizations? 

9. Is there a means for evaluating MIS project 
complexity on an objective basis allowing 
comparison across project and organization 
boundaries? 

Most of the above nine areas of suggested research are 
broad. However, If the present research has brought these Issues 
Into clearer focus so that further research can be conducted 
which will provide meaningful Insights Into the management of 
management Information systems, an important objective has been 



achieved 
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APPENDIX A 
SMIS QUESTIONNAIRE 

One of the most Important areas In the MIS field is 
that of the management of MIS projects. Below are listed a 
number of factors which can influence the overall success of 
MIS projects. We would like to have your opinion concerning 
how these factors influence MIS project success. 

We recognize that "project success" is a vague term. 
For this reason we prefer that you use your own definition and 
frame of reference for making responses. Likewise, "project" 
is vague; so assume that it refers to a development effort on 
the part of EDP and/or user personnel requiring at least six 
man-months to complete. 

Direc tlons : 

Please evaluate each of the thirty-four listed factors 
in terms of the factor: 

"Performance standards employed for analysts 
and programmers" 

In other words, is any given factor cited below - more, less, 
or of equal importance - in determining MIS project success 
than is "performance standards for analysts and programmers." 
Please check the appropriate blank for each factor. 
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Standard factor: ’’Performance standards employed 

for analysts and programmers” 



Importance compared to standard: 

much somewhat about somewhat much 
less less s ame more more 



1) High formal educational level 

of project personnel 

2) Proficiency of project person- 
nel as judged by superiors. • • 

3) Length of experience in the 

organization of project 
personnel . 

4) Systems experience of project 

personnel • 

5) Coordinating ability of project 
leader (superior's evaluation). 

6) Persuasiveness of project 
leader (superior's evaluation). 

7) Low turnover rate of MIS Dept . 

Staff 

8) Low turnover of project 
personnel ........... 

9) High average income level of 

MIS Dept. Staff 

10) High rates of MIS Dept. Staff 

drawn from within the organi- 
zation 

11) High availability of computer 
time for program testing. . . . 

12) Program maintenance and review 

responsibility specified for 
definite period after imple- 
mentation 
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Standard factor: "Performance standards employed 

for analysts and programmers" 

Importance compared to standard: 

much somewhat about somewhat much 

less less same more more 



13) Measurable project objectives 
from conception of the project 

14) Formal project selection 
process used to determine 
which projects to develop. . . 

13) Separation of analysts and 

programmers for large projects 

16) Combination ana lys t /programmer 

for small projects 

17) High level programming lang- 
uage used for project 

18) Formal training program set 
up for user organization • • • 

19) Number of years experience 

for organization with computer- 
ized Information systems * • • ^ 

20) Documentation standards used 
and enforced •••••••••, 

21) Utilization of a project team 

composed of MIS and user 
personnel 

22) Organizational level of top 

computer executive 

23) Source of origination of pro- 
ject (MIS staff or user) . . . _ 

24) Participation by operating 

management In design, formal 
approval of specifications, 
and continual review of pro- 
ject 



Standard factor: 
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"Performance standards employed 
for analysts and programmers" 



Importance compared to standard: 

much somewhat about somewhat much 

less less same more more 



23) Operating management conducts 
periodic management audit of 
MIS function. (Evaluation of 
effectiveness for users) • • • 

26) Utilize existing data base 

versus constructing or 
greatly modifying one 

27) Short-term, minor project 
versus large, complex project. 

28) Planning and accounting for 

all resources throughout 
project development 

29) Utilization of a formal time- 
scheduling technique such as 
PERT for project development . 

30) Use of a formalized and 

regular reporting structure 
on project progress 

31) Ratio of computer hardware 
Investment to total sales 

or operating budget 

32) Overall size of organization 

systems staff 

33) Low degree of overall organi- 
zational change 

34) High centralization of organi- 
zational MIS activities. . . . 
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Please list any other factors which you believe are important to the 
success of a MIS project. 
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APPENDIX B 



:c QUEST IOI.^n^AIRE for study of 

1 SUCCESS/FAILURi: CORRELATES IN MIS 

2 PROGR.\>2iING PROJECTS 



I . Environmental Factors 

A. General Information - Company or Division 
1. Basic Industry; 
a. Manufacturing 

(1) Consumer 

(2) Industrial 



3 




b. 


Wholesale, Retail Trade 






c. 


Transportation 






d. 


Financial 






e . 


Utility 






f 


na^u^e) 




2. 


If 


Manufacturing, Primary Production Technology 






a. 


Unit Job/Small Batch 






b. 


Assembly Line/Mass Production 






c . 


Continuous Process 




3. 


Company or Relevant Division Size Data 


5-11 




a. 


Total Assets 


12-18 




b. 


Total Sales 


19-23 




c. 


Total Employees 


24-26 




d. 


Number of Billed Customers 


27-29 




e. 


Number of Products 


30-31 




f. 


Number of Manufacturing Locations 


32-33 




g- 


Number of Distribution Points 
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34 



35-36 



37-38 



39 



4. Have there been any significant organizational changes in 
last three years? (i.e., mergers, acquisitions, etc.) 



(Circle one) 




Several 






No signifi- 




Minor 




Extensive 


cant changes 




Changes 




Changes 


5 


4 


3 


2 


1 


was the 


ratio 


of computer hardware 


investment 


to last 



year’s total sales or other gross income measure? (If 
rented, monthly rental x 40) 

6. I'That proportion of the operating expenses of the information 

systems function was accounted for by computer hardware last 
year? 

7. Centralization of organization's informaticn system activities, 
a. If a new MIS application is proposed, which of the 

following statements best describes how the organization 

accomplishes the design, analysis and programming 

activities for that project? (Check one) 

All design, analysis, and programming 

tasks are performed by the corporate 

MIS staff regardless of who originated 

the project. 5 

All design, analysis, and programming 

tasks are performed by the corporate 

MIS staff only if the project involves 

organizational segments other than the 

one proposing the project. (I.E., If a 

division has its own systems staff, and 

the project Involves only that division's 

information needs, the division will develop 

its own programs within corporate guidelines.) 4 

General system design always done at the 

corporate level but analysis and programming 

performed by the organizational segment that 

wants the project accomplished. 3 
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The corporate level MIS staff prescribes 
standards and procedures for project 
development, approves or disapproves 
project proposals, and evaluates MIS 
progress and performance. Corporate 
staff does not do the analysis and 

programming. 2 

The corporate MIS staff provides guidance, 
coordination, and assistance to organiza- 
tional segments but all design, analysis 
and programming performed by the organiza- 
tional segment. I 

8. level of the top computer executive: 

a. \Chat is the organizational location of the information 

systems function: 

A separate function reporting directly to top management 

^4 

A separate function reporting directly to administrative 
40 VP or other similar executive. 3 



An organizational component of some other staff 

function such as the Controller. 2 



An organizational component of a line function 

(specify line function) 1 

b. How many hierarchical levels are there beD/een the 

manager of the information systems function and the 

top operating executive? (Circle one) 

0 I 2 3 4 or more 

How many personnel are employed by the company in the systems 



41 



42-44 9. 



design, analysis, and programming activities? 
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45 



II. Project Information 

A. Provided by Project Leader or Information Systems llanager 



1. \Jho originated this project? 

User department 
Top level management 
Information systems staff 
Other (specify) 



A 

’3 

2 

1 



2. Wature of the project: 

a. What functional area of the organization was the 

primary beneficiary of the project? 

(If the project had several primary beneficiaries, 
list them all.) 

b. Project leader’s brief description of the nature of 
the project# 

c. Project coiapletlon date: 

3. Objectives of the Project: (Check most appropriate statement) 

a. Specific measurable objectives were laid doim In 



writing at the outset of the project. ^6 

b. Specific but non-measurable objectives were laid 

doi-m in writing at the outset of the project. ^5 

c. General but clear objectives were laid do\m in 

writing at the outset of the project. 4 

d. General objectives were agreed uoon by 

key parties involved at outset of project but 
46 not committed to writing. 3 



e. Objectives of the project v;ere rather vague at its 
initiation but were wore fully developed later as 

the project progressed. 2 

f. Project objectives were not quite clear to those 

engaged in the project. 1 
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47 



46 



49 



50-51 



52-53 



54 



55 



4. Project leader's ratin^^ of the relative complexity of the 



project: (Check one) 

Tlie most complex project I have been involved with 

in this organization 3 

One of the more complex projects we have undertaken 4 

About average complexity 3 

Less complex than many of the organization's MIS 

projects 2 

One of the least complex liTS projects we have undertaken 1 

5. Was a project team composed of user personnel and information 

systems staff utilized for this project? Yes 1 

No 0 



If yes; 

a. Were user personnel full time or part time on the 

project? Full time 1 

Part tir.e 0 

b. What proportion of the project team was made up of 

user personnel? 

c. Of the total man months devoted to the project, what 

proportion were accounted for by user personnel? 

d. l^en the design and programming effort was completed 

were the user personnel responsible for implementing 

the project in their respective organizational areas? 

Yes ^1 

Wo p 

e. Did the user personnel who worked on the project team 
carry on the liaison with the information systems staff 
after implementation? 

Yes 1 

Wo 0 





I 



-- t !► 




41 



i 
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f. l/hile on tl'.e project team, to whom did user 
personnel report as their superior? 



56 Project leader 3 

Information system 2 

Regular superior 1 



g. By whom were user personnel evaluated? 

Project leader 3 

57 Information system mgr. 2 

Regular superior ^1 



h. In the opinion of the project leader, hov; instrumental 
were user members in contributing to project success? 
(Cneck one) 

Very great contribution — were critical to success 5 

llade some contribution; could not have done so 

well v;ithout them 4 

58 

Their presence on the team didn’t contribute much; 
could have done just as veil without them 3 

Their presence on the team had a slightly negative 
influence, v;e had to train them, etc,, slowed us down 



Their presence on the project team was detrimental 
to project success, created conflict 1 



59-60 


6. 


How many people v;orked directly on the project? 


61-62 




Number of Analysts 


63-64 




Number of Programmers 


65-66 




User Representatives 


67 




Consultants 


68-69 


7. 


How many months elapsed from initiation of the project 
to implementation? 


70-72 


8. 


ilow many man months were spent on the project? 
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CARD n 

9. Were the aiialysis and progranming functions accoi^iplished ’jy 

1 separate groups or by personnel who were coir.blnatlon analysts/ 

programmers? Corah in at ion 1 

Separate groups 0 

10, Systems experience of project personnel; 

2-A a. How i:\any nonths experience in systems work has the 

project leader had? 

b. \Jhat is the mean length of systcris experience (in 

5-6 months) for project personnel, including user 

organization members if applicable? 

c. Provide same information as for 9 (b) above 

7-9 excluding user organization project members. 

d. Uhat proportion of the project staff had two or 
more years of systems experience at the beginning 

10-11 of the project? 

e. Project leader’s appraisal of Impact that prior 
systeuis experience of project staff had on project 
success : 



High experience was critical to project success. 5 

12 Experience contributed somewhat to project success. 4 

Experience in systems work v;as not very Important 

one v;ay or the other. 3 

Experience had mixed effects; the high experience of 
some team methers t;as very important > but v;as offset 
sone\;hat by inexperience of other team i.embers 2 

On balance, the lack of systems experience was 
detrimental to project success. ^1 

11. Project staff turnover; 

a. What was the percentage turnover within the project 

staff during the course of the project? 
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(This should be calculated as follows) : 

, . , . A/D 

1 3-14 -2p — * 

A*the number of individuals added to the project 
staff as replacements for departed members 

D*=the number of individuals who left the project 
staff before the project’s iinplen.entation for 
any reason besides completion of all assi^.ned tasks. 

?=the median number of individuals on the project staff. 

b. Project leader’s appraisal of the effect of personnel turnover 

on project success; (Circle one) 

15 Very Somewhat Made no Contributed 

detrimental detrimental difference to success 

4 3 2 1 

12. Project staff educational level: 

16-17 a. What proportion of the project staff held college 

degrees? 

b. hli<it proportion of the project staff courpleLcu at 

18-19 least r.’O years of college or Junior college? 

c. ^-Tnat was the mean number of years of education for 

20-21 the project staff? 

(12 years for high school j 4 years for college, etc. 

Person with a college degree vrould have 16 years of 
education) 

13. Organizational experience of project staff; 

a. V^aat was the mean number of years experience in this 
22-23 organization for project staff members at the beginning 

of the project? 
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24-25 



26 



27 



28 



29 



b. proportion of the project staff had D^o or icore 

years of experience in this organization at the beginning 

of the project? 

14. Documentation 

a. Were fomal documentation standards specified for or 

applicable to the project? Yes ^1 

No 0 



b. Project leader’s appraisal of importance of the 

documentation standards* (Circle one) 

Made signifi- 
cant contribu- Somex/hat Were just 

tion to success useful make-work 

5 4 3 2 1 

c. Project leader’s appraisal of compliance of the 
project staff with docuroentation standards: 



Project staff followed standards completely; each 
ph;^se of documentaf ion renuired by standards v»as 
completed at the specified time during project 
development. 5 



Project staff followed standards for the most part; 
however, documentation was occasionally lacking in 
quality and timeliness. ^4 



Tried to follow standards but just couldn’t keep up 

with it and get the programming done, too. 3 



Standards were used for guidance mainly not followed 
religiously. 2 



L'id not follow standards; each project meruber did what 
he felt was required. ^1 



d. Row ouch time and effort were devoted to enforcing 
documentation standards? (Circle one) 

Very great Moderate Very little 

time & effort time 6 effort time & effort 

5 4 3 2 1 

e. Project leader’s appraisal of quality of the documentation: 



* •* 
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30 Documentation was of high quality: experienced no 

problems because of inadequate documentation. 4 

Documentation was adequate in most cases, some 
problems of inadequate documentation did arise 
but they were very minor. 3 

Documentation problems were frequent but such problems 
did not seriously affect the success of the projectj 2 

Documentation problems were serious; had a detrimental 
effect on success of the project. 1 

f. If documentation standards were not followed, was this 
a result primarily of; (Check one) 

Unrealistic standards 

Inadequate standards 

31 Lack of time to document 

Lack of willingness to 

document 

Skip 32-33 Other (specify) 



15. Participation by user organization management in design, 
formal approval of specifications, and continual review 
of project; 



34 



a. What was the degree of the user organization's active 
participation throughout the evolution of this project? 
(Circle one) 

They 

Very High {^derate provided what Almost 

great participation amount was asked for none 

5 4 3 2 1 



b. What contribution do you feel user participation made to 
the success of this project? 



User participation was definitely an important contributor 
to project success ^5 



35 


User participation probably contributed somewhat to 
project success 


4 




User participation of no real consequence one way or 
the other 


3 




The lack of meaningful user participation probably 
detracted somewhat from project success. 


2 
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36 



37 



The lack of meaningful ujer participation definitely 

had a detrimental effect on project success. 

c. How would you evaluate the quality of the communications 
between your staff and the user organization on this 
project? (Circle one) 

Highly constructive Somewhat 

and effective Good Adequate inadequate Poor 

5 4 3 2 1 

d. How clearly were user organiza t ions * detailed requirements 
and desires specified at the cutset of the project? 

(Circle one) 

Very clearly Generally Not too 

and explicitly clear Adequate clear Very Vaeue 

5 4 3 1 



33 



39 

Skip 40-41 



e. Was management at the top levels of the user organization 
interested in the project and in directing its evolution? 

(Circle one) 

Extremely interested Interested Disinterested 

and active participation but passive and pensive 

5 4 3 7 1 

f. Were user organizations' desires for products of the 
project incorporated into the project as it developed even if 
doing so entailed changes to what had already been accomplished? 
(Circle one) 

Always Usual 1 y Half the time Seldom Never 

5 4 3 2 1 



16. VTiich statement most nearly matches the situation that existed 
for project control on this project? 



There was a formalized reporting system which required 

that each project member report to the project leader 

his progress against assigned or planned tasks at least 

every month. 5 

42 No formalized reporting system was required by the 

organization but the project leader instituted his own; 
each project member reported his progress against 
assigned or planned tasks to the project leader at 

least every month. U 
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43 



44 



45-46 



There was no regular progress reporting required of 

project members. Uox;ever, the project leader was in 

frequent touch with all project n embers and thus kept 

the project status and progress records himself. 3 

There was no regular progress reporting required of 

project members. Project leader either occasionally 

asked project members how they were progressing or 

left it to project memhers to Inform him if falling 

behind schedule. 2 

Ihere was no task schedule set up against which to 

measure progress of project staff members. 1 

17. What programming language was used for this project? 

18. Were there operating system or other generalized soft\7are 
probleins? (Circle one) 

No problems Minor problems Ilajor problems 

3 2 1 

B . Provided by User Fers onne 1 



1. User organization participation in the project: 

a. How clearly were your organization’s detailed 
requirements and desires specified at the outset 
of the project? (Circle one) 

Very clearly Generally Not too Very 

and explicitly clear Adequately clear vague 

5 4 3 2 1 

b. I'/hat was the degree of your organization’s active 
participation throughout the evolution of this 
project? (Circle one) 



47-48 
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49-50 



51-52 



53-54 



55 



56 



It was our Provided 

project: l’*igh par- Moderate v?hat was Alcost 

very r.reat ticipation amou nt asked for none 

5 4 3 2 1 

c. Did you desire to have more of a voice in specifying 

and approving the products of the project than you 

actually did have? 

Couldn't 

have had Uanted soiiiewhat Definitely did not 

more voice more of a voice have enough voice 

5 4 3 2 1 



d. Hov; vrould you evaluate the quality of the communica- 
tions between your organization and the systems staff 
on this project? (Circle one) 

Highly constructive Somewhat 

and effective Good Adequate inadequate Poor 

5 4 3 2 1 



e. Do you feel this project resulted in what you wanted 
or what the stuff gave you? (Circle one) 



Exactly what 
ve v;anted 
5 



4 



About half 
and half 
3 



2 



\?hat the 
staff gave us 
1 



f. Did you make formal arrangements for liaison between 
the project staff and your personnel? 

Yes ^1 

No 0 



g. Did you assign any person or persons in your 



organization responsibility for coordinating the 
project in your organization? 

Yes ^1 

No ^0 



h. 



Yes 

No 



1 

0 



57 



If answer to (e) was yes, did you relieve 
this person (persons) of normal duties? 



y>^p 



Skip 58-60 
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III . Su ccess Criteria 

(A project Is considered coaiplete v;hen turned over to computer 
operations) 



61 



62 



63-64 



65 



66-67 



68-70 

71-73 



74 



A, Time 

ills manager or project leader 



1 . 



a. 



V/as the project completed within the calendar 
time originally allocated for It? 

Yes 1 

No 0 



b. If completed on time, were additional staff 
resources, over v;hat originally were planned, 
required to do so? 

Yes 1 

No 0 



c. If ans^^er to (b) is yes, what percentage of 
the total project effort In man months vas 
accounted for by these additional resources? 
d* If completed on schedule, unantlclp.‘*ted 



overtime required to do so? 

Yes 1 

No p 

e. If ansvjer to (d) is yes, what percentage of 

total project time in man months was overtime? 

2. What were the actual nan months required to complete the 

project as a percentage of man months originally allocated 
to the project? 

3. l^iat was the actual project completion time as a 

percentage of allocated time? (elapsed time) 

4. a. Did the project leader believe the time originally 

allocated for the project was realistic? 

Yes 

No 0 



b. If the answer to (a) is no, what did the project 
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75 



Card ^^3 



leader believe to be a realistic tine period for 
completion of the project (as a percenta 3 e of the 
time actually allocated - e.g., 120%) 

5. If the time schedule v;as not net, x;hat vas the primary 
reason given by the project leaaer? 

(Answer this question on the back of preceding page.) 

6. Relative to past experience, how successful does the 
project leader feel this project v;as in terms of time 
to complete It? (Circle one) 

Highly Above Below Very 

successful average Average average poor 

5 4 3 2 1 



B. Cost: 



ins Manager or Project Leader 

1. Was the project completed ^dthin the original cost 

1 budget for the project? Yes 1 

No 0 



2-4 



5 



2. What was the actual project cost as a percentage of 

budgeted cost? 

3. If the project exceeded budget, v;hat was the priraary 

% 

reason for this in the opinion of the project leader? 

4. Relative to past experience, hew successful does the 
project leader feel this project v;as in terms of cost? 
(Circle one) 

Highly Above Below Very 

successful average Avera?,e average poor 
5 4 3 2 1 



I 
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6-7 



8-9 



10-11 



12-13 



^ • User Perceptions of Success o f Project : 

For each primary user or^anization- 

For two levels of management in the user organization 

1. liow closely do the products of the project match what you 
wanted and expected from the project? (Circle one) 

Got even 

more than Good Satisfactory Marginal Poor 

e xpected match match match match 

5 4 3 2 1 

2. Do you believe the originally stated objectives for the 
project were, or are in the process of being, satisfied? 
(Circle one) 

For the Not Not very Definitely 



Definitely most part certain well not 



5 


4 


3 


2 1 




Overall, 


how satisfied 


are you with 


the products of the 


project? 


(Circle one) 








Highly 

pleased 


Well 

satisfied 


Products 

acceptable 


Products 

marginal Dissatisfied 


5 


4 


3 


2 


1 


Have there been implementation problems associated 


with 


this project in your organization? 

Very minor Moderate 

No problems problems problems 


(Circle one) 

Considerable 

difficulties 


Very 

serious 

problems 


5 


4 


3 


2 


1 



5. How well has this project been accepted by your organization? 
(Circle one) 










m 



m I 
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14-15 



16-17 



Very 

Good Satisfactory Marginal negat- 

Enthusiastically acceptance acceptance acceptance ively 

5 4 3 2 1 

6. What do you think should have been done to improve the 
effectiveness and value of the project with respect to 
your information needs? 

7. Do you feel this project is completed? Yes 1 

No 0 



Skip 18-21 
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22 



23 



24 



25 



D. Project Leader*s perception of Success of Project : 

1. Do you believe the originally stated objectives for the 
project were, or are in the process of being, satisfied? 
(Circle one) 

For the Not Hot very 

Definitely most part certain veil Ho 

5 4 3 2 1 

2. Overall, hou successful do you think this project v/as? 
(Circle one) 

Extremely Very Moderately Hot 

successful successful successful certain Marginal 

5 4 3 2 1 

3* How well satisfied do you feel the users are with the 

products of this project? (Circle one) 

Highly Hell Somewhat 

pleased satisfied Satisfied dissatisfied dissatisfied 
5 4 3 2 1 

4. VJere there implementation problems associated with this 

project? (Circle one) 

Very 

Very minor Iloderate Considerable serious 

Ho problems problems problems problems problems 

% 5 4 3 2 1 



Skip 26-27 









I 
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20 



29 



30 



E. Coir.puter Operating Cost ; 



iilS llanager or Computer Operations Manager 

1. Has this project caused problems for computer operations? 



(Circle one) 



Very minor Moderate Considerable 
No problems problems problems difficulties 
5 4 3 2 



Very 

serious 

problems 

1 



2. V/ould you say the costs of operation for this project are 
excessive? (Circle one) 

Definitely Probably Not Probably Definitely 

not not certain are are 

5 4 3 2 1 

3. Are you running this project less frequently than the users 
desire? (Circle one) 

Run l^enever Usually run Not Run nuch less Virtually 

desired as desired certain than desired never run 



5 



4 



3 



2 



1 



225 



UNIVERSITY OF 



'Minnesota 



July 9, 1970 



APPENDIX C 

LETTER REQUESTING PARTICIPATION IN STUDY 

MANAGEMENT INFORMATION SYSTEMS RESEARCH CENTER 
SCHOOL OF BUSINESS ADMINISTRATION 
MINNEAPOLIS, MINNESOTA S5455 



Representative's Name 

Pos Itlon 

Firm 

Address 

Dear Mr. Representative: 

One of our MIS doctoral students, Dick Powers, Is doing 
research on the organisational and procedural correlates 
of success with MIS projects. Dick has reached the point 
where he desires to gather data on twenty MIS projects for 
a statistical analysis of several hypotheses he has set up. 

He hopes to collect the required data by Interviewing 
selected people In ten associate firms (two MIS projects per 
firm) during the period 1 October to 31 December, 1970. 

Within the next several days, Dick will be contacting you 
to arrange an appointment of about thirty minutes duration 
to explain more fully the research he Is doing, and to discuss 
what will be Involved on your part should you agree to partici- 
pate. I believe the research Is most worthwhile and hope you 
will elect to participate. 

Since Dick will be doing all of the data collection and 
analysis himself. It Is appropriate that you know sooiethlng 
about his qualifications and background. To this end, a 
brief biographical sketch Is attached. 

Thank you for your continued support and cooperation. 

Sincerely, 



Gary W. Dickson 
Associate Professor 



GWD/Jls 

Enclosure 
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Brief Biographical Sketch of Richard F. Powers 



Dick Powers is currently a doctoral student majoring in 
management information systems at the University of Mixmesota. 
He entered the MIS program in the fall of 1968 and has now 
completed all course work required for the PhD. Dick is a 
lieutenant commander in the Supply Corps of the U.S. Navy, 
and is being sponsored in his doctoral studies by the Navy. 

He holds a BA degree from Rice University, 1958, and an MBA 
from Michigan State University, 1965. 

Since his commissioning as an ensign in 1958, Dick has held 
the following positions up to the time he entered the MIS 
program at Minnesota: 



1959-1960 


Supply and Disbursing Officer, USS MASSEY 
(DD-778) 


1961 


Aide to the Commanding Officer, Naval 
Supply Center, Norfolk, Va. 


1962-1963 


Director of Data Processing, Naval 
Supply Center, Norfolk, Va. 


1964 


Control Division Officer, Royal Canadian 
Naval Supply Depot, Victoria, British 
Columbia 


1965 


MBA program, Michigan State University 


1966-1968 


Director of Computer Training, Navy 
Supply Corps School, Athens, Ga. 
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APPENDIX D 



BRIEF DESCRIPTION OF STUDY 



MANAGEMENT INFORMATION SYSTEMS RESEARCH CENTER 
SCHOOL OF BUSINESS ADMINISTRATION 
MINNEAPOLIS, MINNESOTA 5S4SS 



Research on the Correlates of MIS Project Success 



Nature of the Research ; 

An empirical investigation of sixteen organizational 
and procedural factors hypothesized to be correlates of success 
with MIS projects. 

Methodology ; 

Study two MIS projects in each of ten organizations 
with as wide representation across industries as possible. 

Criteria of Success for a Project ; 

Time 

Cost 

User satisfaction 
Computer operating problems 

Data Collection ; 

By structured interview that follows a pre-tested 
questionnaire. The responses are anchored, and multiple 
scales are used wherever possible to Improve reliability 
of the results. 

For each project selected for investigation the 
following individuals will be interviewed with the 
Interviews requiring approximately the amount of time 
shown; 



In addition to the above interviews for each of two 
projects in each organization, interviews will be conducted 
with the following individuals to collect computer operat- 
ing information or information not specifically related to 
one project: 



Individual 
Project Leader 
User Management, Level 1 
User Management, Level 2 



2 Hours 
% Hour 
% Hour 



Time 



Project Total 
For Two Projects 



3 Hours 
6 Hours 






# , .. 







m 


















11 # 



I 



I' * ^ 
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MISRC Associate ^ Hour 
Manager of MIS ^ Hour 
Manager of Computer Operations ^ Hour 



Hours 



Total Interviewing Time 

per Organization 7^ Hours 



Data Analysis : 

All data collected will be analyzed statistically 
using non-parametric techniques such as X^. Qualitative 
analysis of various moderator variables will also be made* 

Expected Results of the Research ; 

1. Confirmation or rejection of the numerous 
prescriptions appearing in information systems literature. 

2. A ranking or other classification of those 
organizational and procedural factors related to MIS project 
success in the operational, real-world environment. This 
ranking or classification will have more general meaning for 
MIS management than previous case studies* 

3. The manager should be able to judge more 
accurately which organizational and procedural factors are 
most critical to project success given a particular set of 
circumstances . 

Assurances to the Participating Organization ; 

1. Reported results of the research will be made 
available to the participant company. 

2. No company names nor non-public financial 
information will be reported without the express prior 
permission of the company. 

3. No information provided by an individual will 
be divulged to any other person or be attributed to the 
individual in the report without the express prior consent 
of that individual. 

4. All information gathered will be retained 
personally by the researcher at all times* 
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Variable 

Number 



1 



2 



3 



4 



APPENDIX E 

VARIABLES INCLUDED IN THE STUDY 



Variable 

Name 



Source 

Page Question(s) 



Variable Description and Scoring 

ASSETS 206 

Collected as a measure of relative 
size of organizations in the sample. 

Asset data were not available for 
three of the organizations. 

SALES 206 

Total sales or other gross revenue 
measure for the past fiscal year. 

Not available for two of the organi- 
zations in the sample. 

EMPLOYEES 206 

Number of employees in domestic 
operations of each organization. 

The only measure of organization 
size which was available for all 
organizations in the sample. 

CUSTOMERS 206 

Total billed customers. Collected 
as a measure of relative organiza- 
tion size. Not available for two 
organizations . 

ORGANIZATION CHANGE 207 

Degree of organizational change 
during past three years; scaled 
from *'no change*' at high end (5) 
to **ex tensive change** at low end 
(1). Pertains to changes affect- 
ing the whole organization as 
opposed to in tr a functional realign- 
ments . 
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6 HARDWARE INVESTMENT/SALES 207 

Measure of investment in computing 
hardware and supporting equipment 
relative to total revenue for past 
fiscal year. If owned, equipment 
valued at replacement cost or 
current list price for identical 
or comparable equipment. If rented, 
monthly rental price multiplied by 40. 

7 HARDWARE EXPENSE/BUDGET 207 

Represents the proportion of last 
fiscal year's budget for the 
information systems function that 
was accounted for by hardware cost 
(depreciation and/or rental). 

8 CENTRALIZATION 207 

Centralization of analysis, design 
and programming activities in the 
organization. Scaled from complete 
centralization (5) to decentraliza- 
tion (1). This variable was not 
adequately tested by the present 
sample; eight organizations were 
scored at 5 level and remaining two 
were scored at 4 level. The latter 
two were not at the 5 level solely 
because of small operations research 
groups which did their own design 
and programming. For this reason. 

Variable 8 is not tabled with other 
variables in the study. 

9 LEVEL OF INFORMATION SYSTEMS MANAGER 208 

Number of hierarchical levels between 
the manager of the information systems 
function and the toj^ operating execu- 
tive of the organization. Scaled 
from no intervening levels (0) to 
four or more intervening levels (4). 

The lower the score on this variable, 
the higher the information systems 
function in the organization. 
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10 NUMBER IN SYSTEMS & PROGRAMMING 208 

This variable, the total number of 
personnel engaged In systems analy- 
sis, design, and programming activi- 
ties within the organization, is 
used as the numbeator for computing 
the value of Variable 61. 

11 ORIGINATOR 209 

This variable identifies the primary 
originator of a project being studied. 

It is scaled from user origination 
(4) to information systems staff 
origination (2); there were no 
projects reported as "other** (1). 

In terms of rankings, high rank is 
given to user originated projects; 
middle rank to those of top manage- 
ment origin; and low rank to systems 
staff originated projects. 

12 MEASURABLE PROJECT OBJECTIVES 209 

The project leader's perception of 
nature and clarity of project objec- 
tives. Scaled from highly specific 
and measurable objectives in writ- 
ing (6) to very vague objectives (1). 

13 COMPLEXITY 210 

Project leader’s perception of the 
complexity of this project as compared 
with others the organization has under- 
taken. The higher the scale value for 
this variable, the greater the complex- 
ity, with most complex scored 5 and 
least complex scored 1. 

14 PROJECT TEAM 210 

Dichotomous variable scored 1 if a 
project team including user personnel 
developed the project, and scored 0 
if there was not a project team com- 
prised, in part, of user personnel. 
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15 USER MAN MONTHS/TOTAL MAN MONTHS 210 

The proportion of man months spent on 
the project accounted for by user 
personnel. Variable 15 value will 
be 0 for all projects where value of 
Variable 14 is 0, indicating a project 
team including users was not employed. 

16 USER CONTRIBUTION TO TEAM 211 

Project leader's perception of the 
contribution of user members of the 
project team to project success. 

A value for only 13 projects that 
used team. Not included in the 
s ta tis tical analys is • 

17 NUMBER ON PROJECT 211 

Total number of personnel, including 
users and consultants, if any, who 
worked directly on the project at 
any time. 

18 NUMBER OF ANALYSTS 211 

Number of analysts who worked on 
project. Not Included in the 
statistical analysis. 

19 NUMBER OF PROGRAMMERS 211 

Number of programners who worked on 
project. Not included in the 
statistical analysis. 

20 NUMBER OF USERS 211 

Number of user personnel, if any, 
who worked directly on project. 

Not included in the statistical 
analysis . 

21 ELAPSED MONTHS 211 

Number of months included in period 
from formal start of project to pro- 
ject completion; project completion 
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deflned as date when project turned 
over to computer operations for 
production running. 

22 MAN MONTHS 211 8 

Number of roan months spent on project 
by all personnel directly involved 
with project, including analysts, 
programmers, user members of project 
team, and consultants, if any. 

23 C(»1BINATI0N ANALYST/ PROGRAMMER 212 9 

Dichotomous variable scored 1 if 
combination analys t/programners 
were used on the project, and 
scored 0 if the analysis and 
programming tasks were performed by 
separate individuals or groups. 

24 SYSTEMS EXPERIENCE OF PROJECT LEADER 212 10a 

Total number of months systems 
experience the project leader 
possessed at the start of the 
project. Systems experience 
defined to Include programming or 
systems work performed outside of 
a data processing organization, such 
as methods and time studies. This 
variable not used in the statistical 
analysis; Variable 27 used as the 
systems experience variable in all 
reported cross -tabulations . Variable 
24 was highly correlated with Variable 
27, the tau value being .418, which 
is significant at the .014 level 
( two-tailed) . 

25 MEAN SYSTEMS EXPERIENCE OF PROJECT 

PERSONNEL INCLUDING USERS 212 10b 

Mean number of months systems exper- 
ience for entire project team, 
including user personnel and consul- 
tants, if any. Comments under Variable 
24 concerning definition of systems 
experience and tabulation apply 
fully to Variable 25. Tau value for 
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Variable 25 vs. Variable 27 was 
.596, which is significant at the 
.0006 level (two-tailed). 

26 MEAN SYSTEMS EXPERIENCE OF PROJECT 

PERSONNEL FROM MIS STAFF ONLY 212 

Mean number of months systems experi- 
ence for those personnel from the 
information systems function who 
worked on the project. This value 
the same as the value for Variable 
25 for those seven projects where a 
team not used. Comments under 
Variable 24 concerning definition 
of systems experience and tabulation 
apply fully to Variable 26. Tau 
value for Variable 26 vs. Variable 
27 was .575, which is significant 
at the .0004 level (two-tailed). 

27 TWO OR MORE YEARS SYSTEMS EXPERIENCE 212 

Proportion of the project staff, 
including users and consultants, if 
any, with two or more years systems 
experience. Systems experience 
defined exactly as for Variables 
24-26. This measure of systems 
experience considered to contain the 
least distortion from extreme values. 

28 IMPACT OF SYSTEMS EXPERIENCE 212 

Project leader's opinion of the 
impact that prior systems experience 
of project personnel had on project 
success. Scaled from highly critical 
to success (5) to a lack of systems 
experience detrimental to success (1). 

This variable not tabled in the 
statistical analysis. 

29 TURNOVER 212 

The percentage turnover of the project 
staff. Any person who left the 
project staff at any time prior to 
completion of the project was considered 
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as contributing to turnover unless 
that person’s assigned tasks had 
been completed. If a person who 
left the project prematurely was 
replaced^ the replacement was also 
considered as a factor in the numera- 
tor of the turnover computation. 

The denominator of the turnover 
computation was the ''normal** or median 
number of personnel on the project 
staff multiplied by two. 

30 COLLEGE DEGREE 213 12a 

Proportion of project staff that 
held a college degree. Although 
not used as the primary measure of 
the level of education. Variable 30 
did make a contribution to the 
interpretation of certain relation- 
ships, and for this reason it has 
been cross -tabula ted with other 
variables where significant 
relationships exis ted . 

31 TWO YEARS COLLEGE 213 12b 

Proportion of project staff that 
had completed at least two years 
of college. This proportion 
included all of those who held 
college degrees, and was, therefore, 
very strongly associated with 
Variable 30. Since Variable 31 
contributed nothing to the study 
which was not picked up by using 
Variables 30 and 32, it is not 
tabled in the statistical analysis. 

32 MEAN YEARS FORMAL EDUCATION 213 12c 

Mean number of years formal educa- 
tion for the project staff, including 
users and consultants, if any. This 
variable used as primary measure of 
level of education for project staff. 
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33 MEAN YEARS ORGANIZATION EXPERIENCE 213 

Mean number of years experience in 
the organization (firm, not function) 
studied for project staff members. 

Not included in the statistical 
analysis since Variable 34 believed 
to be a less distorted measure of 
organization experience. 

34 TWO OR MORE YEARS ORGANIZATION 

EXPERIENCE 214 

Proportion of the project staff, 
including users and consultants, 
if any, possessing two or more 
years experience in the organiza- 
tion (firm, not function) studied. 

This variable the primary measure 
of organization experience. 

35 DOCUMENTATION STANDARDS 214 

Dichotomous variable scored 1 if 
documentation standards were 
prescribed for the project, and 
scored 0 if such standards were 
not prescribed. 

36 QUALITY OF DOCUMENTATION 214 

Project leader's appraisal of the 
quality of documentation for the 
project. Scaled from high quality 
(4) to serious problems with 
documentation (1). 

37 STANDARDS APPRAISAL 214 

The sum of two separate perceptions 
by the project leader; how important 
he felt the documentation standards 
were plus how well the project staff 
complied with the standards. Since 
Variable 37 made no contribution to 
the analysis of documentation stan- 
dards or quality, it is not tabled. 
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38 SPECIFICITY OF USER REQUIREMENTS— PL 216 

Project leader*s perception of the 
clarity and specificity of user 
requirements and desires at the 
beginning of the project. Scaled 
from very clear and specific (5) 
to very vague (1). 

39 USER MANAGEMENT INTEREST 216 

Project leader*s perception of the 
interest and participation of top 
level user management in the develop- 
ment of the project. Scaled from 
very active participation (5) to 
d is in teres ted (1 ) . 

40 CHANGED AS REQUESTED 216 

Project leader’s perception of the 
response by the project staff to 
user requests for changes in any 
aspect of the project during the 
development period for the project. 

Such changes may have entailed 
merely output format modifications 
or may have required major logic 
alterations. Scaled from always 
made requested changes (5) to 
never made such changes (1). If 
no changes of any kind were requested 
by the user, scored 5. 

41 USER PARTICIPATION— PL 215 

Project leader’s perception of the 
level of user participation in the 
project. An aggregate variable 
representing the sum of three separ- 
ate items. Scaled from very great 
participation (15) to very little (3). 

42 PROJECT CONTROL 216 

Project leader’s rating of the type 
and frequency of progress reporting 
for the project. Scaled from formal- 
ized, monthly progress reporting (5) 
to no progress reporting (1). 
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43 HIGH LEVEL PROGRAMMING LANGUAGE 2L7 

Projects programmed in COBOL were 
scored 3; those programmed in FORTRAN 
were scored 2; and those programmed 
in some assembly language were scored !• 

For the purposes of this study, COBOL 
was considered a higher level language 
than FORTRAN. 

44 GENERALIZED SOFTWARE PROBLEMS 217 

Project leader's perception of the 
existence and seriousness of operating 
system or other generalized software 
problems with respect to this project. 
Scaled from no problems (3) to major 
problems (1). 

45 SPECIFICITY OF USER REQUIREMENTS — 

USER 217 

Users* perception of the clarity and 
specificity of user requirements and 
desires at the beginning of the 
project. Scaled from very clear and 
explicit (50) to very vague (10).* 

46 USER PARTICIPATION— USER 217 

218 

Users* perception of the level of 
user participation in the project. 

An aggregate variable representing 
the sum of four separate items. 

Scaled from very great participation 
(200) to very little participation (40).* 

47 ON TIME 219 

A dichotomous variable scored 1 if 
the project was completed within the 
calendar time originally estimated, 
and scored 0 otherwise. Variable 47 
not used in cross-tabulations since 
Variable 48 was the primary time 
success variable used in the study. 
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48 ACTUAL TIME/ESTIMATED TIME 

The ratio of actual calendar months 
spent on the rpoject over calendar 
months estimated for the project. 

This variable is the criterion 
variable for project success in 
terms of time. 

49 TIME SUCCESS— PL 

Project leader’s opinion of the 
success of this project, in terms 
of time to complete it, relative 
to his experience with actual time/ 
estimated time for other projects. 

Scaled from highly successful (5) 
to very poor (1). 

50 WITHIN BUDGET 

A dichotomous variable scored 1 
if the project was completed with- 
in the original cost budget for 
the project, and scored 0 other- 
wise. Variable 50 not used in 
cross-tabulations since Variable 
51 was the primary cost success 
variable used in the study. 

51 ACTUAL COST/ESTIMATED COST 

The ratio of actual cost of the 
project over estimated or budgeted 
cost for the project. This variable 
is the criterion variable for success 
in terms of cost. Where cost data 
were not available, this ratio 
computed based on actual and estimated 
man months for the project, along 
with an estimated factor for computer 
test time. 

52 COST SUCCESS— PL 220 4 

Project leader’s opinion of the 
success of this project, in terms 
of cost, relative to his experience 
with actual cost/estimated cost for 
other projects. Scaled from highly 
successful (5) to very poor (1). 
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53 IMPLEMENTATION PROBLEMS— USER 221 

Users* perception of the problems 
experienced in implementing the 
project in their organizations. 

Scaled from no problems (50) to 
very serious problems (10).* 

Although this variable is actually 
a component of Variable 54, it is 
also treated separately for compara- 
tive purposes. 

54 USER SATISFACTION— USER 221 

Users* perception of the success of 
the project in terms of five sepa- 
rate factors which were summed to 
derive one overall measure of user 
satisfaction. Scaled from high 
user satisfaction (250) to low 
user satisfaction (50).* This 
variable is the criterion variable 
for success in terms of user satis- 
faction. 

55 SATISFACTION OF OB JECTIVES— PL 223 

Project leader's perception of the 
degree to which project objectives 
were satisfied. Scaled from defin- 
itely satisfied (5) to not satisfied 
( 1 ). 

56 PROJECT SUCCESS— PL 223 

Project leader's global perception 
of the overall success of the 
project. Scaled from extremely 
successful (5) to marginally 
successful (1). 

57 USER SATISFACTION— PL 223 

Project leader's perception of 
user satisfaction with the 
products of the project. Scaled 
from users highly pleased (5) 
to users dissatisfied (1), 
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58 IMPLEMENTATION PROBLEMS- -PL 223 4 

Project leader’s percertion of the 
problems encountered in implementing 
the project* Scaled from no problems 
(5) to very serious problems (1). 

59 COMPUTER OPERATIONS PROBLEMS 224 1 

Perception of the manager of computer 
operations, or his superior in some 
cases, of the problems caused by the 
project for computer operations. 

Scaled from no problems (5) to very 
serious problems (1). This variable 
is the primary criterion for project 
success in terms of computer operations. 

60 COMPUTER OPERATIONS COST 224 2 



Perception of the manager of computer 
operations, or his superior in some 
cases, of the computer operating 
cost for the project. Scaled from 
cost definitely not excessive (5) to 
cost definitely excessive (1). 

61 SYSTEMS STAFF /TOTAL EMPLOYEES 

A measure of the relative size 
of the organization’s systems 
staff (analysts and programmers). 
Computed by dividing the total 
number of analysts and programmers 
(Variable 10) by the total number 
of domestic employees of the 
organization (Variable 3). 



Computed 

from 

Variables 3 
and 10 



* Note: All variables representing users’ perceptions 

were scaled up by 10^ to avoid using fractions 
where there were two or more responses per 
item which were averaged to derive one value 
for that item. Such scaling, of course, had 
no bearing on the relative rankings which were 
used in the analyses. 
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APPENDIX F 

COMPUTING FORMULAS USED*^ 



The following procedures and formulas were used in 
computing the values of tau-B, tau-C, S, variance of S, and 
the normal deviate of S: 

1) All observations on two variables being cross- 
classified were entered into a naturally ordered 
table of the following type: 

j»l to c columns 
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In the above example case, there are five possible 
levels for one variable, and these are represented by 
the columns. There are seven possible values for the 
second variable and these are represented by the rows. 
Any one observation of the total N observations must 
fall in one of the cells of the resulting array. Thus, 



Most of what is presented in this appendix is taken from 
the appendix of UMST 590, the computing program used for the 
data analysis, which, in turn, is from Kendall (1962). 
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the entry in each cell is merely the number of 
observations falling in that cell> or having the 
values of variables one and two that intersect in 
that cell. 



2) S = P • Q 
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3) For square tables (where rSc), Kendall's tau-B was 
computed as follows: 



tau-B s S 

V%N(N-1)-T V%N(N-1)-U 

where T and U are factors representing the number 
of ties in the ranking of each variable, and are 
computed as follows: 

T = 1/2^^ n (n -1) 

1.1 • 

U s 1/2 £’n. .(n. -1) 
jsl 

4) For rectangular tables (where rf^c), Kendall's tau-C 
was computed as follows: 

tau-C s 2S 

2(lmil\ 

N V m / 

where m is the smaller of r or c. 
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5) The normal deviate of S can be computed only after 
the variance of the S statistic has been determined 
as follows: 



VAR S a 1 
18 
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6) The normal deviate of S can next be computed by: 

2(s) « S* 

V Var S 

^Before computing the normal deviate, S is 
corrected for continuity as follows: 

a) for a 2 X 2 table, js| reduced by 
unless S < ^N, in which case S s Oo 

b) for all other tables, js | is reduced by 1 
These corrections for continuity are made to 
adjust the distribution of S closer to the norma 



distribution where there are ties in either or 
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both of the rankings being cross-classified. 
However, where the extent of tied ranks is 
very great, the correction in (b) above may 
not be enough. This is particularly true 
where one ranking is a dichotomy and the other 
ranking has ties of varying extents in it. 

In these last cases, an actual distribution of 
S for each set of variables cross-classified 
would need to be determined to arrive at the 
continuity correction required.^ 



For discussions and proofs of the correction for 
continuity applied to S under various cases of tied ranks, 
see Burr (1960), Kendall (1947; 1962), Sillltto (1947), and 
Whitfield (1947), 
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APPENDIX G 

COMMENTS OF USER MANAGEMENT 

Each user interviewed was given the opportunity at 
the end of the interview to make whatever conments he felt 
were germane to the project being studied. Specifically, each 
user was asked what he felt should have been done to improve 
the effectiveness and value of the project with respect to his 
information needs. Further, the user was asked to elaborate 
on any aspects of the project which he felt required additional 
development work. 

The comments included in this appendix provide some 
flavor of the attitudes of user managers toward the projects 
studied. Since verbatim notes were not taken by the researcher, 
the comments presented here are best described as “paraphrased 
user comments**. However, the comments are presented in the form 
of statements by the users rather than descriptive narratives 
of the researcher. 

1. Problems that were apparent from the beginning 
have never been corrected. There has been 
virtually no follow-through by the project 
staff to help us work with the system. We 
have also been unhappy with the validity of 
the input data; we don't consider the reports 
we're getting to be very reliable. 
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2. We have learned a great deal from the 
experience of working with the system. 

We now feel there Is too much historical 
Information provided In the outputs; 
completed transactions should be purged 
from the files so that they do not appear 
In our reports. Also, there has been a 
lack of training for the people preparing 
the Input data. This has resulted in too 
many Input errors which makes us suspicious 
of the validity of the outputs. 

3. There was too much pressure from top manage- 
ment to get something going in a hurry to 
meet an immediate problem. This led to 
inadequate planning and preparation. I 
think we are trying to do too much with the 
system; we should reduce the scope of this 
project and focus just on the heart of the 
problem, rather than trying to do everything 
for everyone with one project. 

4. We want a good deal more now than we envisioned 
needing when the system was designed. The 
original outputs were fine, based on what we 
said we wanted, but we have been unable to 

get changes made that we feel are necessary now. 
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Another problem was that the salesmen who 
provide the input were never involved in 
system design. As a consequence, they 
cannot see any results from the coding 
they are doing--they see no specific outcome 
and tend to feel they are wasting their time. 

5. This is a very valuable application. The 
opportunity to evolve with it and to incor- 
porate needed changes in phase two was 
crucial to our getting useful outputs. 

6. We are just getting too much paper. We 
need reports oriented to the needs of the 
individual decision maker. 

7. From our standpoint, the system is not 
worth too much. The production engineers 
who are supposed to use the system had 
little or no voice in designing it. Higher 
level managers were involved in defining 
the system, which we believe resulted in 
outputs more oriented to accounting needs 
than to production requirements. The 
reports are too voluminous and hard to 

use; there should be more exception reporting. 

8. I get a lot of performance figures, but these 
are meaningless to me without some standard 
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or scale for comparison. What Is good, bad, 
acceptable? I don't believe there Is a 
real means for a manager In my position to 
use the system— to really be an effective 
Influence on actions taken. Although some 
of the other managers were Involved In 
developing the system, I don't believe I 
had enough voice In It. 

9. We don't get the output In a format that 
Is usable. Our analysts still have to 
rearrange the data and prepare the required 
reports by hand. However, I think the 
blame for this situation falls on us--we 
did not state explicitly enough what was 
needed. 

10. Eighty to ninety percent of our Information 

requires output formatting of a certain tjrpe. 

We don't get that type of output at all. The 

need for this type of output format was very 

clearly specified In the original project 
1 

request. 

11. It seems that the original objectives were 
satisfied well enough, but It is apparent now 

^Comments 9 and 10 came from two different users of the 
project outputs In the same organization. 
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that we did not really know what we wanted 
or needed when we started. We have grown 
with experience with the present system--we now 
have a better grasp of what information we 
should be getting but are unable to get it 
with the present system. Among the specific 
shortcomings, as I see them, are: the 

information is not now current or accurate 
enough to make the kinds of decisions we 
should be making; reports are too detailed 
and VO luminous --we need much more analytical 
reports that satisfy our information needs 
directly without further manipulation; and 
too many people are involved in data prepara- 
tion with no effective input quality control. 

12. The form required to be filled out for an 
inquiry run is rather complex. As a result, 
some potential users have not used the system 
as they could. I guess this may partly stem 
from the fact that all potential user areas 
were not well enough represented and involved 
in original system definition. 

13. Due to the way the project was developed, and 
the way it is now used, the field sales force 



is very opposed to the forecasting system. 
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They were not consulted in the development 
efforts, and they have no input to the processing 
cycle now. However, I guess there has been a 
time constraint that precluded our including 
the field sales force in the processing cycle. 

14. Very simply, we did not ask for enough; we 
did not specify what we wanted clearly enough; 
and the result is that we are still doing too 
much manually. 

15. I can think of a couple of things that are 
probably related. First, the documentation has 
been inadequate in that changes to programs 
have often not been reflected in documentation 
nor communicated to users. Second, the field 
training of operating people has been inadequate 
which, I think, accounts for some of their resist- 
ance to the system. 

16. There are too many inaccuracies in the data-- 
inadequate input controls and data validation. 
Also, the reports are too detailed and difficult 
to use for vice presidents and assistant vice 
presidents. Finally, we have had implementation 
problems. We are not getting the reports where 
and when we should in some cases. This, combined 
with inaccuracies, is frustrating. 
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